-- Logs begin at Wed 2022-06-29 09:13:28 UTC, end at Wed 2022-06-29 10:48:07 UTC. -- Jun 29 09:43:43.149585 dv-cinder-768105-16-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 29 09:43:43.158498 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 29 09:43:43.160843 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 29 09:43:43 2022] *** Jun 29 09:43:43.160843 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 29 09:43:43.160843 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 29 09:43:43.160843 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: nodename: dv-cinder-768105-16-iscsi Jun 29 09:43:43.160843 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: machine: x86_64 Jun 29 09:43:43.160843 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: clock source: unix Jun 29 09:43:43.160843 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: pcre jit disabled Jun 29 09:43:43.160843 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: detected number of CPU cores: 4 Jun 29 09:43:43.160843 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: current working directory: / Jun 29 09:43:43.160843 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: detected binary path: /usr/bin/uwsgi-core Jun 29 09:43:43.160843 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: your processes number limit is 47680 Jun 29 09:43:43.161410 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: your memory page size is 4096 bytes Jun 29 09:43:43.161410 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: detected max file descriptor number: 2048 Jun 29 09:43:43.161410 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: lock engine: pthread robust mutexes Jun 29 09:43:43.161410 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: thunder lock: enabled Jun 29 09:43:43.161410 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 29 09:43:43.161410 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 29 09:43:43.204777 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: Python main interpreter initialized at 0x5650e4fcc9c0 Jun 29 09:43:43.204777 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: python threads support enabled Jun 29 09:43:43.204777 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: your server socket listen backlog is limited to 100 connections Jun 29 09:43:43.204777 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: your mercy for graceful operations on workers is 90 seconds Jun 29 09:43:43.205170 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: mapped 403077 bytes (393 KB) for 2 cores Jun 29 09:43:43.205246 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: *** Operational MODE: preforking *** Jun 29 09:43:43.205493 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: *** uWSGI is running in multiple interpreter mode *** Jun 29 09:43:43.205493 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: spawned uWSGI master process (pid: 80165) Jun 29 09:43:43.205705 dv-cinder-768105-16-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 29 09:43:43.210602 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: spawned uWSGI worker 1 (pid: 80176, cores: 1) Jun 29 09:43:43.210602 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: spawned uWSGI worker 2 (pid: 80177, cores: 1) Jun 29 09:43:43.210602 dv-cinder-768105-16-iscsi devstack@keystone.service[80165]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 29 09:43:48.595712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 09:43:48.595712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: from cryptography.utils import int_from_bytes Jun 29 09:43:48.596554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 09:43:48.596554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: from cryptography.utils import int_from_bytes Jun 29 09:43:48.596663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 09:43:48.596663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: from cryptography.utils import int_from_bytes Jun 29 09:43:48.596663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 09:43:48.596663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: from cryptography.utils import int_from_bytes Jun 29 09:43:48.644759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 29 09:43:48.644983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 29 09:43:48.645111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] command line args: [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 29 09:43:48.645205 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 29 09:43:48.645352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 29 09:43:48.645502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 29 09:43:48.645502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ================================================================================ {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 29 09:43:48.645644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] command line args: [] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 29 09:43:48.645826 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] admin_token = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.645958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 29 09:43:48.645958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ================================================================================ {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 29 09:43:48.646098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] config_dir = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.646164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.646222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] admin_token = **** {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.646357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] config_source = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.646485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] config_dir = [] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.646590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] debug = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.647107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.647187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] config_source = [] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] debug = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648464 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648609 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648917 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.648979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_config_append = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] list_limit = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] log_config_append = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_dir = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_file = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] log_dir = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] log_file = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.649992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650175 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650678 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650780 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.650909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.651013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.651114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.651247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.651343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.651478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.651573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.651725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.651812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.651970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.652247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.652330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.652465 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] publish_errors = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.652698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.652779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.652915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] publish_errors = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.653038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.653150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.653258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.653365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.653481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.653590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.653716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.653837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] standard_threads = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.653948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.654058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.654177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.654341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.654455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] standard_threads = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.654604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.654726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.654838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] use_journal = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.654948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.655070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] use_json = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.655183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.655293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] use_stderr = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.655409 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] use_journal = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.655520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] use_syslog = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.655638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] use_json = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.655758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.655870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] use_stderr = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.655979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.656089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] use_syslog = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.656199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.656317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 09:43:48.656434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.656552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.656665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.656777 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.656887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.657004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.657123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.657235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.657351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.657480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.657593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.657711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.657830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.657941 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.658051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.658160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.658269 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.658387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.658499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.658622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.658734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.658845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.658955 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.659078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.659190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.659301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.659417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.659528 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.659837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.659959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.660071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.660071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.660284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.660395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.660395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.660617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.660728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.660844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.660957 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.661073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.661193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.661310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.661422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.661531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.661641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.661751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.661751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.661973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.662086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.662202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.662320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.662431 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.662549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.662699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.662818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.662930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.663041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.663159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.663277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.663388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.663498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.663607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.663718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.663835 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.663947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.664063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.664185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.664296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.664406 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.664524 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.664636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.664746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.664866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.664983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.665094 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.665281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.665396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.665506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.665625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.665742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.665858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.665981 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.666098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.666210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.666320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.666429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.666544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.666701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.666814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.666925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.667034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.667143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.667254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.667360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.667455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.667548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.667640 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.667733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.667831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.667940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.668036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.668129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.668222 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.668316 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.668440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.668569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.668683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.668794 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.668904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.669018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.669128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.669247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.669360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.669477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.669595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.669707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.669817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.669940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.670061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.670174 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.670285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.670407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.670520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.670690 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.670805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.670917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.671037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.671150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.671261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.671378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.671508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.671619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.671721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.671831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.671942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.672052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.672164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.672287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.672399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.672511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.672622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.672622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.672854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.672967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.673078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.673189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.673316 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.673429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.673549 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.673662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.673773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.673883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.673993 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.674104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.674226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.674339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.674451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.674619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.674735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.674851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.675163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.675280 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.675392 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.675503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.675616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.675728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.675848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.675967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.676080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.676192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.676303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.676303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.676590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.676707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.676819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.676938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.677052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.677163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.677282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.677407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.677532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.677646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.677764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.677877 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.677995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.678107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.678227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.678340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.678451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.678586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.678701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.678813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.678937 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.679051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.679171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.679285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.679396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.679508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.679627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.679741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.679853 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.679965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.680075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.680193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.680314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.external = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.680314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.680532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.680532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] auth.external = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.680741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.680864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.681009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.681009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.password = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.681239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.681355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.token = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.681467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.681578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.681690 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] auth.password = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.681809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.681924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] auth.token = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.682036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.682161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.682275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.682387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.682507 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.682637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.682761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.682875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.682988 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.683101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.683221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.683335 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.683446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.683559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.683660 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.683781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] credential.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.683902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.684017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.684129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.684244 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.684357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.684469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.684592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.684706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.684818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.684929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.685041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.685153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.685273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.685387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.685505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.685618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.685729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.685841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.685960 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.686074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.686187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.686299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.686411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.686522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.686703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.686821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.686941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.687055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.687170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.687283 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.687408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.687523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] federation.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.687636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.687748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.687860 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.687973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.688093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.688209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.688321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.688434 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.688546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.688658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.688779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.688893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.689013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.689126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.689241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.689353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.689473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.689587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.689700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.689812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.689923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.690045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.690235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.690354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.690466 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.690593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.690708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.690833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.690954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.691067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.691180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.691276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.691388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.691509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.691623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.691735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.691847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.691958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.692076 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.692197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.692311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.692424 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.692537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.692648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.692760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.692880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.692994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.693106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.693217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.693330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.693443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.693563 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.693677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.693789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.693901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.694013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.694125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.694249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.694363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.694475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.694620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.694735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.694848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.694972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.695087 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.695199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.695317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.695431 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.695544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.695664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.695778 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.695891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.696002 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.696115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.696227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.696348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.696461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.696574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.696686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.696799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.696916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.697038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.697152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.697265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.697377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.697488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.697600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.697721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.697835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.697948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.698060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.698172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.698290 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.698415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.698530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.698660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.698772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.698884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.698996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.699120 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.699234 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.699346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.699457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.699569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.699681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.699803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.699916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.699916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.700127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.700239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.700351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.700471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.700585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.700697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.700809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.700921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.701032 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.701197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.701314 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.701427 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.701545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.701659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.701770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.701893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.702007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.702119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.702231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.702343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.702463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.702596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.702711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.702823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.702936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.703047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.703171 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.703286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.703434 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.703551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.703664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.703775 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.703895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.704011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.704123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.704238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.704351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.704463 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.704582 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.704708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.704822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.704933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.705047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.705158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.705279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.705392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.705505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.705616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.705728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.705846 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.705967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.706095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.706287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.706400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.706518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.706665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.706846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.706966 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.707081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.707194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.707306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.707418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.707538 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.707658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.707772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.707884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.707997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.708110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.708230 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.708345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.708457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.708603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.708745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.708858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.708979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.709093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.709207 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.709319 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.709431 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.709542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.709662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.709776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.709888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.710001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.710112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.710225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.710345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.710488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.710666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.710782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.710894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.711006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.711130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] resource.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.711246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.711360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.711472 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.711585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.711698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.711818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.711932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.712045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.712157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.712269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.712383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.712573 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.712690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] role.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.712803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.712915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] role.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.712915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.713126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.713247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.713362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.713362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] role.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.713573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.713687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] role.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.713799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.713923 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.714039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.714152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.714267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.714380 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.714502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.714632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.714746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.714746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.714958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.715070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.715194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.715310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.715423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.715536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.715649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.715761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.715882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.715996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.716108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.716108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.716319 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.716432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.716554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.716668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.716781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.716990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.717109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.717109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.717329 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.717444 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.717558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.717670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.717783 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.717895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.718016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.718131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.718244 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.718356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.718468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.718600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.718770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.718888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.719001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.719113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.719226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.719338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.719459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.719573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.719687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.719799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.719911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.720031 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.720145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.720257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.720369 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.720481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.720593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.720714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.720828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.720940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.721053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.721171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.721285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.721405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.721520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.721631 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.721850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] token.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.721968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.722102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.722228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.722344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.722456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.722675 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.722794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.722907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.723032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.723149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.723261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.723374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.723486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.723598 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.723718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.723718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.723934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.724046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.724158 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.724270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.724401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.724515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.724626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.724738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.724850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.724962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.connection = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.725087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.725209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.725331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.725430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.725544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.725656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.726133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.726245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.726366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.726480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.726609 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.connection = **** {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.726738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.726861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.726974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.727089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.727209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.727324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.727436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.727548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.727661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.727773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.727893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.728007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.728121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.728233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.728345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.728458 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.728578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.728693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.728805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.728805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729364 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729589 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 29 09:43:48.729701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 09:43:48.729701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80176) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 29 09:43:48.758401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.758659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.758859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.759797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.760033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.760262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.767415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.769027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.795848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.796050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.797525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.797735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.942645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.942911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.992798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.993013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.993177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:48.993306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.058694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.058920 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.059165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.059353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.059594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.059719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.059799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.059862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.060117 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.060338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.060553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.060752 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.085261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.085493 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.085572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.085649 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.085809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.085901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.085964 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.086095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.086274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.086351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.086414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.086475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.086719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.086887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.097761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.098081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.110878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.111116 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.111320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.111449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.111528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.111591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.111665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.111818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.111903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.112052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.112145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.112330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.112527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.112780 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80176) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 09:43:49.129794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.129956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 09:43:49.130218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.130368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.130452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 09:43:49.130529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 09:43:49.130777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.130901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.130978 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 09:43:49.131063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.131213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.131296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.131506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.131632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.131723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.132001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.142242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.142517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.142805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.142928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.143136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.143264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.143477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.143779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.151531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.152262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.154802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.155070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.155343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.155572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.155717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.155904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.156030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.156166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.156245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.156340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.156439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.156635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.156759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.156896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.156976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.157108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.157188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.157389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.157513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.157671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.157935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.158183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.184317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.185033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.185932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.186209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.186694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.186975 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.191776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.192052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.192324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.192468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.192607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.193436 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.200083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.200340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.200703 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.200987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.204799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.205079 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.205354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.205477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.205582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.205746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.205842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.206046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.206172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.206331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.206421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.206662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.206788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.206980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.207168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.207458 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.225150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.225423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.226612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.226881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.230425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.231391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.234060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.234348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.235241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.235539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.239618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.240857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.243186 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.244498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.246792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.247084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.247363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.247650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.247934 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.248167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.248292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.248422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.248735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.249020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.249306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.249596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.267255 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.267527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.267792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.268045 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.268841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.269133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.269405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.269697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.276318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.277578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.277993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.278266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.279292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.279570 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.281501 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.281778 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.282049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.282323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.282606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.282747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.283046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.283338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.283613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.283906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.295852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.296124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.296397 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.296670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.296960 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.297249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.297527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.297811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.298097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.298376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.298713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.298991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.300186 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.300478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.316964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.318337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.320243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.321621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.323538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.324959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.325258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.326714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.326950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.327227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.327504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.328338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.328619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.328907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.334841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.336352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.336479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.337967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.339623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.339911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.340298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.340570 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.341236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.341541 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.341933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.342223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.349573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.349850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.350131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.351707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.351996 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.352280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.358231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.358500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.358794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.359063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.359321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.359694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.359967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.360193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80176) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 09:43:49.360319 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.360523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.360653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.360793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.360987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.361120 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.361246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.361246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.361553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.361683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.361809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.361937 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.362064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.362200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.362510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.362812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.363261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.363528 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.363782 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.364052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80176) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 09:43:49.399157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5650e4fcc9c0 pid: 80177 (default app) Jun 29 09:43:49.402031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5650e4fcc9c0 pid: 80176 (default app) Jun 29 09:43:49.402671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dee5b-6463-4a52-bb3d-7e17e0d17fd1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:43:49.402964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dee5b-6463-4a52-bb3d-7e17e0d17fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:43:49.403237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6dee5b-6463-4a52-bb3d-7e17e0d17fd1 None None] PATH_INFO: `/v3` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:43:49.407468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Wed Jun 29 09:43:49 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:43:57.215107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a11a5-9cf3-4efd-bc60-4860ce8e4a25 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:43:57.215698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a11a5-9cf3-4efd-bc60-4860ce8e4a25 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:43:57.215698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a11a5-9cf3-4efd-bc60-4860ce8e4a25 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:43:57.220214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:43:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:43:57.227610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f7a91-8202-4df2-b8d3-60c69bbcace9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:43:57.228133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f7a91-8202-4df2-b8d3-60c69bbcace9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:43:57.228463 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f7a91-8202-4df2-b8d3-60c69bbcace9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:43:57.315829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: INFO dbcounter [None req-998f7a91-8202-4df2-b8d3-60c69bbcace9 None None] Registered counter for database keystone Jun 29 09:43:57.337415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG oslo_db.sqlalchemy.engines [None req-998f7a91-8202-4df2-b8d3-60c69bbcace9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80177) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 29 09:43:57.344991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writer thread running {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 29 09:43:57.521635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG passlib.handlers.bcrypt [None req-998f7a91-8202-4df2-b8d3-60c69bbcace9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80177) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 29 09:43:57.522045 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG passlib.handlers.bcrypt [None req-998f7a91-8202-4df2-b8d3-60c69bbcace9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80177) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 29 09:43:57.578084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-998f7a91-8202-4df2-b8d3-60c69bbcace9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:43:57.579647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-998f7a91-8202-4df2-b8d3-60c69bbcace9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:43:57.724020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:43:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 29 09:43:57.912251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a11a5-9cf3-4efd-bc60-4860ce8e4a25 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:43:57.983470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: INFO dbcounter [None req-d25a11a5-9cf3-4efd-bc60-4860ce8e4a25 None None] Registered counter for database keystone Jun 29 09:43:58.003630 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG oslo_db.sqlalchemy.engines [None req-d25a11a5-9cf3-4efd-bc60-4860ce8e4a25 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80176) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 29 09:43:58.011146 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writer thread running {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 29 09:43:58.132391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a11a5-9cf3-4efd-bc60-4860ce8e4a25 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:43:58.137178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1164ad-b5cd-473e-9a59-bf865e54d2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:43:58.137847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1164ad-b5cd-473e-9a59-bf865e54d2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:43:58.138125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1164ad-b5cd-473e-9a59-bf865e54d2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:43:58.138401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1164ad-b5cd-473e-9a59-bf865e54d2b0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:43:58.147251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1164ad-b5cd-473e-9a59-bf865e54d2b0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:43:58.199556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7b1164ad-b5cd-473e-9a59-bf865e54d2b0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 29 09:43:58.199890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1164ad-b5cd-473e-9a59-bf865e54d2b0 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:43:58.210917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Jun 29 09:43:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 09:43:58.217052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f7a91-8202-4df2-b8d3-60c69bbcace9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:43:58.232705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f7a91-8202-4df2-b8d3-60c69bbcace9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:43:58.238711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9093434-b591-490f-8a41-5e050ad0de9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:43:58.239860 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9093434-b591-490f-8a41-5e050ad0de9c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:43:58.240143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9093434-b591-490f-8a41-5e050ad0de9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:43:58.240417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9093434-b591-490f-8a41-5e050ad0de9c admin admin] PATH_INFO: `/v3/projects/d79cb8b2f46e418d955fc3be77ed8c7a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:43:58.242071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9093434-b591-490f-8a41-5e050ad0de9c admin admin] RBAC: Authorizing `identity:get_project(project_id=d79cb8b2f46e418d955fc3be77ed8c7a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:43:58.295169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e9093434-b591-490f-8a41-5e050ad0de9c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 29 09:43:58.295477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9093434-b591-490f-8a41-5e050ad0de9c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:43:58.300464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Wed Jun 29 09:43:58 2022] GET /identity/v3/projects/d79cb8b2f46e418d955fc3be77ed8c7a => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:00.328155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27760-235b-4456-a84e-c859ae5d88b0 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:00.328682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27760-235b-4456-a84e-c859ae5d88b0 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:00.328682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27760-235b-4456-a84e-c859ae5d88b0 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:00.333250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:00.339027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee46c9-b3f3-4b8f-ad3d-0958b166e6f2 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:00.339486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee46c9-b3f3-4b8f-ad3d-0958b166e6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:00.339576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee46c9-b3f3-4b8f-ad3d-0958b166e6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:00.360133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4bee46c9-b3f3-4b8f-ad3d-0958b166e6f2 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:00.361656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4bee46c9-b3f3-4b8f-ad3d-0958b166e6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:00.371602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 29 09:44:00.394600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b27760-235b-4456-a84e-c859ae5d88b0 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:00.408306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b27760-235b-4456-a84e-c859ae5d88b0 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:00.413073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f975f50-fd05-4a24-9c4a-6a9a036cfcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:00.413721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f975f50-fd05-4a24-9c4a-6a9a036cfcee admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:00.414001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f975f50-fd05-4a24-9c4a-6a9a036cfcee admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:00.414277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f975f50-fd05-4a24-9c4a-6a9a036cfcee admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:00.422413 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f975f50-fd05-4a24-9c4a-6a9a036cfcee admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:00.424231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f975f50-fd05-4a24-9c4a-6a9a036cfcee admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:00.433104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Jun 29 09:44:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 09:44:00.439016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee46c9-b3f3-4b8f-ad3d-0958b166e6f2 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:00.448743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee46c9-b3f3-4b8f-ad3d-0958b166e6f2 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:00.454527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600674a-5ad5-4962-95ff-6b394caab767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:00.455452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600674a-5ad5-4962-95ff-6b394caab767 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:00.455732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600674a-5ad5-4962-95ff-6b394caab767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:00.456377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600674a-5ad5-4962-95ff-6b394caab767 admin admin] PATH_INFO: `/v3/users/a021026b1cd24ba98923ebdb4a56f71f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:00.457531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600674a-5ad5-4962-95ff-6b394caab767 admin admin] RBAC: Authorizing `identity:get_user(user_id=a021026b1cd24ba98923ebdb4a56f71f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:00.459992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600674a-5ad5-4962-95ff-6b394caab767 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:00.463072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Wed Jun 29 09:44:00 2022] GET /identity/v3/users/a021026b1cd24ba98923ebdb4a56f71f => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:02.536383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268f6a8-fe56-4d14-8c6d-c8d40c31b95d None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:02.536933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268f6a8-fe56-4d14-8c6d-c8d40c31b95d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:02.536933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268f6a8-fe56-4d14-8c6d-c8d40c31b95d None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:02.538290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:02.543719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17802-d85d-462a-9c15-403d56f1322a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:02.544179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17802-d85d-462a-9c15-403d56f1322a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:02.544267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17802-d85d-462a-9c15-403d56f1322a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:02.561587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f3f17802-d85d-462a-9c15-403d56f1322a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:02.562812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f3f17802-d85d-462a-9c15-403d56f1322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:02.565953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9778594-2115-4db5-83bb-7c301903dde5 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:02.566239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9778594-2115-4db5-83bb-7c301903dde5 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:02.566519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9778594-2115-4db5-83bb-7c301903dde5 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:02.568050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:02.573827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2b277-0dfc-4a7a-8950-6fe2d069585e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:02.574119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2b277-0dfc-4a7a-8950-6fe2d069585e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:02.574396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2b277-0dfc-4a7a-8950-6fe2d069585e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:02.617902 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG passlib.handlers.bcrypt [None req-a4e2b277-0dfc-4a7a-8950-6fe2d069585e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80176) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 29 09:44:02.618329 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG passlib.handlers.bcrypt [None req-a4e2b277-0dfc-4a7a-8950-6fe2d069585e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80176) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 29 09:44:02.648494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:02.665285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a4e2b277-0dfc-4a7a-8950-6fe2d069585e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:02.666930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a4e2b277-0dfc-4a7a-8950-6fe2d069585e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:02.671621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f17802-d85d-462a-9c15-403d56f1322a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:02.698115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:02.772916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:02.783235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f17802-d85d-462a-9c15-403d56f1322a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:02.795774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2b277-0dfc-4a7a-8950-6fe2d069585e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:02.819560 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dc32e-bce0-4bb7-9952-95d0a1c532f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:02.820382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dc32e-bce0-4bb7-9952-95d0a1c532f5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:02.820667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dc32e-bce0-4bb7-9952-95d0a1c532f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:02.820946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dc32e-bce0-4bb7-9952-95d0a1c532f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:02.833159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dc32e-bce0-4bb7-9952-95d0a1c532f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-6e9dc32e-bce0-4bb7-9952-95d0a1c532f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:02.844332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:02.845239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:02.846057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:44:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:02.852010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dc32e-bce0-4bb7-9952-95d0a1c532f5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:02.902582 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2b277-0dfc-4a7a-8950-6fe2d069585e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:02.905672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:02.935258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:02.945055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39a5d4-9416-4816-bdfe-752b17a36e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:02.946073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39a5d4-9416-4816-bdfe-752b17a36e45 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:02.946432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39a5d4-9416-4816-bdfe-752b17a36e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:02.946804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39a5d4-9416-4816-bdfe-752b17a36e45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:02.954741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dc32e-bce0-4bb7-9952-95d0a1c532f5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:02.990079 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca645d-d3d4-4ea8-9fde-8939c6da5b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:02.990774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca645d-d3d4-4ea8-9fde-8939c6da5b70 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:02.991056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca645d-d3d4-4ea8-9fde-8939c6da5b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:02.991333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca645d-d3d4-4ea8-9fde-8939c6da5b70 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:02.992100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca645d-d3d4-4ea8-9fde-8939c6da5b70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:02.992408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca645d-d3d4-4ea8-9fde-8939c6da5b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:02.993867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca645d-d3d4-4ea8-9fde-8939c6da5b70 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:03.006768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:03.012575 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca645d-d3d4-4ea8-9fde-8939c6da5b70 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:03.019242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39a5d4-9416-4816-bdfe-752b17a36e45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:03.020823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39a5d4-9416-4816-bdfe-752b17a36e45 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:03.120596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca645d-d3d4-4ea8-9fde-8939c6da5b70 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:03.121247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 09:44:03.126955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39a5d4-9416-4816-bdfe-752b17a36e45 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:03.129938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:03.155927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102eb306-98b7-4d9c-b6df-40ddbfbd3d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:03.156819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb306-98b7-4d9c-b6df-40ddbfbd3d74 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:03.157085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb306-98b7-4d9c-b6df-40ddbfbd3d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:03.157355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb306-98b7-4d9c-b6df-40ddbfbd3d74 None admin] PATH_INFO: `/v3/users/a021026b1cd24ba98923ebdb4a56f71f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:03.158623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102eb306-98b7-4d9c-b6df-40ddbfbd3d74 None admin] RBAC: Authorizing `identity:get_user(user_id=a021026b1cd24ba98923ebdb4a56f71f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:03.160213 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102eb306-98b7-4d9c-b6df-40ddbfbd3d74 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:03.161882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:03 2022] GET /identity/v3/users/a021026b1cd24ba98923ebdb4a56f71f => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:03.167402 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102eb306-98b7-4d9c-b6df-40ddbfbd3d74 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:03.182461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:03.230712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39a5d4-9416-4816-bdfe-752b17a36e45 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:03.264402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edebd63-fb75-4a65-ba00-061ff660abdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:03.265213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edebd63-fb75-4a65-ba00-061ff660abdf None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:03.265495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edebd63-fb75-4a65-ba00-061ff660abdf None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:03.265769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edebd63-fb75-4a65-ba00-061ff660abdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:03.268803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102eb306-98b7-4d9c-b6df-40ddbfbd3d74 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:03.286810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edebd63-fb75-4a65-ba00-061ff660abdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:03.288305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edebd63-fb75-4a65-ba00-061ff660abdf None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:03.305673 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2144b0-8aa0-4537-8dca-88f26767cb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:03.306396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2144b0-8aa0-4537-8dca-88f26767cb74 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:03.306709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2144b0-8aa0-4537-8dca-88f26767cb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:03.306975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2144b0-8aa0-4537-8dca-88f26767cb74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:03.308230 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2144b0-8aa0-4537-8dca-88f26767cb74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-5edebd63-fb75-4a65-ba00-061ff660abdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:44:03.309428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:03.310257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:03.311092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:44:03.311092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:44:03.311092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:44:03.311092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:03.311294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2144b0-8aa0-4537-8dca-88f26767cb74 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:03.311357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:03.311677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:03.317858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edebd63-fb75-4a65-ba00-061ff660abdf None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:03.318159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2144b0-8aa0-4537-8dca-88f26767cb74 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:03.375976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:03.402370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:03.421366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edebd63-fb75-4a65-ba00-061ff660abdf None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:03.421909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2144b0-8aa0-4537-8dca-88f26767cb74 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:03.454871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c213be-668b-41f6-8d03-87c1677b7db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:03.455548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c213be-668b-41f6-8d03-87c1677b7db7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:03.455836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd82c57-7650-42dc-b0f1-84bcd0e0b8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:03.455981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c213be-668b-41f6-8d03-87c1677b7db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:03.456160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c213be-668b-41f6-8d03-87c1677b7db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:03.457034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c213be-668b-41f6-8d03-87c1677b7db7 None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.domain.id=default, user.id=a021026b1cd24ba98923ebdb4a56f71f) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:03.457188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82c57-7650-42dc-b0f1-84bcd0e0b8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:03.457343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c213be-668b-41f6-8d03-87c1677b7db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:03.457479 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82c57-7650-42dc-b0f1-84bcd0e0b8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:03.457750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82c57-7650-42dc-b0f1-84bcd0e0b8c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:03.458495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd82c57-7650-42dc-b0f1-84bcd0e0b8c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:03.458840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd82c57-7650-42dc-b0f1-84bcd0e0b8c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:03.458926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c213be-668b-41f6-8d03-87c1677b7db7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:03.460271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd82c57-7650-42dc-b0f1-84bcd0e0b8c1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:03.466875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Jun 29 09:44:03 2022] GET /identity/v3/role_assignments?user.id=a021026b1cd24ba98923ebdb4a56f71f&scope.domain.id=default&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:03.481204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:05.847042 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83583d31-44f2-427d-b2f4-38f0cc8d8394 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:05.848142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83583d31-44f2-427d-b2f4-38f0cc8d8394 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:05.848720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83583d31-44f2-427d-b2f4-38f0cc8d8394 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:05.851046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:05.856414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b199a-4b25-4eb8-9451-ad0950197d22 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:05.857241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b199a-4b25-4eb8-9451-ad0950197d22 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:05.857705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b199a-4b25-4eb8-9451-ad0950197d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:05.887626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-829b199a-4b25-4eb8-9451-ad0950197d22 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:05.889215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-829b199a-4b25-4eb8-9451-ad0950197d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:05.989051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:06.019735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83583d31-44f2-427d-b2f4-38f0cc8d8394 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:06.022296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb5b53-365f-4e1f-ade8-f74e074e9afe None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:06.022622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb5b53-365f-4e1f-ade8-f74e074e9afe None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:06.022900 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb5b53-365f-4e1f-ade8-f74e074e9afe None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:06.024440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 12/24] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:06.029721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64190b0-db40-4615-ab3f-3c44d43a4d3d None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:06.030001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64190b0-db40-4615-ab3f-3c44d43a4d3d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:06.030286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64190b0-db40-4615-ab3f-3c44d43a4d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:06.059672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a64190b0-db40-4615-ab3f-3c44d43a4d3d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:06.061017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a64190b0-db40-4615-ab3f-3c44d43a4d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:06.159196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:06.211138 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 13/25] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:06.219437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73936f-9c9b-46c6-a055-33e22fd21037 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:06.219754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73936f-9c9b-46c6-a055-33e22fd21037 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:06.220013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73936f-9c9b-46c6-a055-33e22fd21037 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:06.221439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 14/26] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:06.228006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378236e-ab31-43c3-81d8-8b4007d31835 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:06.228290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378236e-ab31-43c3-81d8-8b4007d31835 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:06.228655 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378236e-ab31-43c3-81d8-8b4007d31835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:06.230294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83583d31-44f2-427d-b2f4-38f0cc8d8394 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:06.266656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-5378236e-ab31-43c3-81d8-8b4007d31835 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:06.267879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5378236e-ab31-43c3-81d8-8b4007d31835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:06.290041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8731215-7f15-484d-bdc3-df15dd296970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:06.291150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8731215-7f15-484d-bdc3-df15dd296970 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:06.291562 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8731215-7f15-484d-bdc3-df15dd296970 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:06.292057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8731215-7f15-484d-bdc3-df15dd296970 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:06.305192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8731215-7f15-484d-bdc3-df15dd296970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f8731215-7f15-484d-bdc3-df15dd296970 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:06.313404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:06.314343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:06.315126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:06.317512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 13/27] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:44:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:06.318518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8731215-7f15-484d-bdc3-df15dd296970 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:06.331505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:06.392942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 15/28] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:06.393936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378236e-ab31-43c3-81d8-8b4007d31835 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:06.425924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8731215-7f15-484d-bdc3-df15dd296970 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:06.461247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0ab69-1470-4b05-a683-33597cec3ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:06.462436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0ab69-1470-4b05-a683-33597cec3ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:06.462961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0ab69-1470-4b05-a683-33597cec3ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:06.463425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0ab69-1470-4b05-a683-33597cec3ae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:06.475179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0ab69-1470-4b05-a683-33597cec3ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:06.477306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0ab69-1470-4b05-a683-33597cec3ae6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-4ee0ab69-1470-4b05-a683-33597cec3ae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:44:06.497650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:06.498510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:06.499326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:44:06.499326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:44:06.499326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:44:06.499326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:06.501261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 14/29] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:06.502213 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0ab69-1470-4b05-a683-33597cec3ae6 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:06.556157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378236e-ab31-43c3-81d8-8b4007d31835 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:06.566887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:06.625187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365f7c9-cc95-46dd-a8f6-beb4d94295f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:06.625419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:06.625869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365f7c9-cc95-46dd-a8f6-beb4d94295f2 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:06.626311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365f7c9-cc95-46dd-a8f6-beb4d94295f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:06.627447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365f7c9-cc95-46dd-a8f6-beb4d94295f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:06.628395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365f7c9-cc95-46dd-a8f6-beb4d94295f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:06.628668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365f7c9-cc95-46dd-a8f6-beb4d94295f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:06.630077 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365f7c9-cc95-46dd-a8f6-beb4d94295f2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:06.647498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 16/30] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:06.648629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365f7c9-cc95-46dd-a8f6-beb4d94295f2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:06.709923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0ab69-1470-4b05-a683-33597cec3ae6 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:06.760306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2b036-d401-44ea-a42f-01d7edf07aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:06.761513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2b036-d401-44ea-a42f-01d7edf07aa1 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:06.762167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2b036-d401-44ea-a42f-01d7edf07aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:06.762645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2b036-d401-44ea-a42f-01d7edf07aa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:06.764262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2b036-d401-44ea-a42f-01d7edf07aa1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:06.766724 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2b036-d401-44ea-a42f-01d7edf07aa1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:06.768837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365f7c9-cc95-46dd-a8f6-beb4d94295f2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:06.816180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9290a71-8e1a-4c32-9fa2-64d86bc9d51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:06.817097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9290a71-8e1a-4c32-9fa2-64d86bc9d51f None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:06.817385 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9290a71-8e1a-4c32-9fa2-64d86bc9d51f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:06.817755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9290a71-8e1a-4c32-9fa2-64d86bc9d51f None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:06.818512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9290a71-8e1a-4c32-9fa2-64d86bc9d51f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:06.818854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9290a71-8e1a-4c32-9fa2-64d86bc9d51f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:06.820261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9290a71-8e1a-4c32-9fa2-64d86bc9d51f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:06.833535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 17/31] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:06.834475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9290a71-8e1a-4c32-9fa2-64d86bc9d51f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:06.868981 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:06.913148 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 15/32] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jun 29 09:44:06 2022] POST /identity/v3/roles => generated 219 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:44:06.915509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3d6db-b9ad-46b6-b685-c309b63fc433 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:06.916037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3d6db-b9ad-46b6-b685-c309b63fc433 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:06.916510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3d6db-b9ad-46b6-b685-c309b63fc433 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:06.923221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 16/33] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:06.924572 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3d6db-b9ad-46b6-b685-c309b63fc433 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:06.964397 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9290a71-8e1a-4c32-9fa2-64d86bc9d51f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:07.013419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b2f11-4f92-4bc5-b2cd-3396aab50d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:07.014362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2f11-4f92-4bc5-b2cd-3396aab50d1c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:07.014679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2f11-4f92-4bc5-b2cd-3396aab50d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:07.014964 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b2f11-4f92-4bc5-b2cd-3396aab50d1c None admin] PATH_INFO: `/v3/users/a021026b1cd24ba98923ebdb4a56f71f` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:07.017974 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b2f11-4f92-4bc5-b2cd-3396aab50d1c None admin] RBAC: Authorizing `identity:get_user(user_id=a021026b1cd24ba98923ebdb4a56f71f)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:07.020123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b2f11-4f92-4bc5-b2cd-3396aab50d1c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:07.023084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 18/34] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:06 2022] GET /identity/v3/users/a021026b1cd24ba98923ebdb4a56f71f => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:07.024840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b7a68-6b19-4bd8-9ab3-8d75b572124a None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:07.025689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b7a68-6b19-4bd8-9ab3-8d75b572124a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:07.026116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b7a68-6b19-4bd8-9ab3-8d75b572124a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:07.048517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3d6db-b9ad-46b6-b685-c309b63fc433 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:07.059822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-eb5b7a68-6b19-4bd8-9ab3-8d75b572124a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:07.062152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-eb5b7a68-6b19-4bd8-9ab3-8d75b572124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:07.071719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:07.093821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796eb1c6-cab3-4ddb-a598-89f7ea528688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:07.094781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eb1c6-cab3-4ddb-a598-89f7ea528688 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:07.095205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eb1c6-cab3-4ddb-a598-89f7ea528688 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:07.095686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eb1c6-cab3-4ddb-a598-89f7ea528688 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:07.096871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796eb1c6-cab3-4ddb-a598-89f7ea528688 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:07.098888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796eb1c6-cab3-4ddb-a598-89f7ea528688 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:07.140021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 17/35] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:44:06 2022] POST /identity/v3/projects => generated 301 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:44:07.141831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796eb1c6-cab3-4ddb-a598-89f7ea528688 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:07.167640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 19/36] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:07.211173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b7a68-6b19-4bd8-9ab3-8d75b572124a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:07.244865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:07.302256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796eb1c6-cab3-4ddb-a598-89f7ea528688 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:07.332309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b7a68-6b19-4bd8-9ab3-8d75b572124a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:07.341855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9ec12-d02c-4107-81cc-a1d69176718d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:07.344143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9ec12-d02c-4107-81cc-a1d69176718d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:07.344655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9ec12-d02c-4107-81cc-a1d69176718d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:07.345122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9ec12-d02c-4107-81cc-a1d69176718d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:07.346627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9ec12-d02c-4107-81cc-a1d69176718d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:07.348387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9ec12-d02c-4107-81cc-a1d69176718d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:07.350278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 18/37] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:07.356638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9ec12-d02c-4107-81cc-a1d69176718d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:07.375080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbae3c5-2cb1-419b-a22b-d3cb6bc69d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:07.376261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbae3c5-2cb1-419b-a22b-d3cb6bc69d55 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:07.376780 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbae3c5-2cb1-419b-a22b-d3cb6bc69d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:07.379391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbae3c5-2cb1-419b-a22b-d3cb6bc69d55 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:07.380553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbae3c5-2cb1-419b-a22b-d3cb6bc69d55 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:07.382440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbae3c5-2cb1-419b-a22b-d3cb6bc69d55 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:07.406904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:07.491136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9ec12-d02c-4107-81cc-a1d69176718d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:07.512449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 20/38] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jun 29 09:44:07 2022] POST /identity/v3/roles => generated 225 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:44:07.560004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ab63b-e68f-4533-9b42-8462bc8b132a None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:07.560710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ab63b-e68f-4533-9b42-8462bc8b132a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:07.561215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ab63b-e68f-4533-9b42-8462bc8b132a None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:07.563024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 21/39] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:07.570605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f10ff-93fe-4801-89ec-1e13eb6c6489 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:07.571411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f10ff-93fe-4801-89ec-1e13eb6c6489 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:07.572193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f10ff-93fe-4801-89ec-1e13eb6c6489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:07.593468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-727f10ff-93fe-4801-89ec-1e13eb6c6489 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:07.595615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-727f10ff-93fe-4801-89ec-1e13eb6c6489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:07.666236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441795ff-82af-41b2-ae69-891882b77151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:07.667926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441795ff-82af-41b2-ae69-891882b77151 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:07.668227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441795ff-82af-41b2-ae69-891882b77151 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:07.668510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441795ff-82af-41b2-ae69-891882b77151 None admin] PATH_INFO: `/v3/domains/default/users/a021026b1cd24ba98923ebdb4a56f71f/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:07.670765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441795ff-82af-41b2-ae69-891882b77151 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a021026b1cd24ba98923ebdb4a56f71f, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:07.672445 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441795ff-82af-41b2-ae69-891882b77151 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:07.687845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 22/40] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:07.690334 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec250a3-1e05-4fe8-a5b8-e45005a6220c None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:07.691014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec250a3-1e05-4fe8-a5b8-e45005a6220c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:07.691309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec250a3-1e05-4fe8-a5b8-e45005a6220c None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:07.693115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 23/41] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:07.698135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 19/42] 10.222.0.52 () {58 vars in 1361 bytes} [Wed Jun 29 09:44:07 2022] PUT /identity/v3/domains/default/users/a021026b1cd24ba98923ebdb4a56f71f/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:44:07.702599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848cf32-6b7a-420e-b6c4-d9639cfdeaa6 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:07.702599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848cf32-6b7a-420e-b6c4-d9639cfdeaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:07.702599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848cf32-6b7a-420e-b6c4-d9639cfdeaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:07.720337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-e848cf32-6b7a-420e-b6c4-d9639cfdeaa6 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:07.721620 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e848cf32-6b7a-420e-b6c4-d9639cfdeaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:07.724416 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441795ff-82af-41b2-ae69-891882b77151 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:07.740975 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:07.817443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 24/43] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:07.841651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848cf32-6b7a-420e-b6c4-d9639cfdeaa6 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:07.871993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:07.883562 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441795ff-82af-41b2-ae69-891882b77151 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:07.921990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12289fac-ce33-452f-afa2-823e59464a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:07.922976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12289fac-ce33-452f-afa2-823e59464a7c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:07.923740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12289fac-ce33-452f-afa2-823e59464a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:07.924019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12289fac-ce33-452f-afa2-823e59464a7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:07.925988 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12289fac-ce33-452f-afa2-823e59464a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:07.928424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12289fac-ce33-452f-afa2-823e59464a7c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:07.930339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 20/44] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:07.935685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12289fac-ce33-452f-afa2-823e59464a7c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:07.961951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848cf32-6b7a-420e-b6c4-d9639cfdeaa6 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:08.000838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2f835-3680-46c8-944c-f2227514c124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:08.001651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2f835-3680-46c8-944c-f2227514c124 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:08.002048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2f835-3680-46c8-944c-f2227514c124 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:08.002342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2f835-3680-46c8-944c-f2227514c124 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:08.003199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2f835-3680-46c8-944c-f2227514c124 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:08.004737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2f835-3680-46c8-944c-f2227514c124 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:08.024094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 25/45] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jun 29 09:44:07 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:44:08.049385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12289fac-ce33-452f-afa2-823e59464a7c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:08.092918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f0f71-43f8-4836-9071-108d5d4fa535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:08.093579 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f0f71-43f8-4836-9071-108d5d4fa535 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:08.093866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f0f71-43f8-4836-9071-108d5d4fa535 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:08.094142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f0f71-43f8-4836-9071-108d5d4fa535 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:08.094929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f0f71-43f8-4836-9071-108d5d4fa535 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:08.096523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f0f71-43f8-4836-9071-108d5d4fa535 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:08.127260 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 21/46] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:44:07 2022] POST /identity/v3/projects => generated 312 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:44:09.795108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4da52f-3c3d-4a1d-bfce-25d26a98d15b None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:09.795635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4da52f-3c3d-4a1d-bfce-25d26a98d15b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:09.795743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4da52f-3c3d-4a1d-bfce-25d26a98d15b None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:09.797128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 26/47] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:09.802932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c527ab9-202f-4f41-912a-0478342a4aef None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:09.803395 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c527ab9-202f-4f41-912a-0478342a4aef None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:09.803497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c527ab9-202f-4f41-912a-0478342a4aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:09.822431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4c527ab9-202f-4f41-912a-0478342a4aef None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:09.824004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4c527ab9-202f-4f41-912a-0478342a4aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:09.836369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:09.898044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 22/48] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:09.921881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4da52f-3c3d-4a1d-bfce-25d26a98d15b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:09.931784 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:10.034804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4da52f-3c3d-4a1d-bfce-25d26a98d15b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:10.073655 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167bca3-9cd5-4baa-b28f-01408bb928ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:10.074471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167bca3-9cd5-4baa-b28f-01408bb928ed None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:10.074788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167bca3-9cd5-4baa-b28f-01408bb928ed None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:10.075069 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167bca3-9cd5-4baa-b28f-01408bb928ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:10.087922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167bca3-9cd5-4baa-b28f-01408bb928ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-a167bca3-9cd5-4baa-b28f-01408bb928ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:10.096879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:10.097857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:10.098622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 27/49] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:44:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:10.104597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c527ab9-202f-4f41-912a-0478342a4aef None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:10.181605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0e5b8-5858-4a32-a410-353fafe8c175 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:10.181921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0e5b8-5858-4a32-a410-353fafe8c175 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:10.182200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0e5b8-5858-4a32-a410-353fafe8c175 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:10.183809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 28/50] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:10.188762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11b5ff-6788-4bd1-ac95-db5e9f90e726 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:10.189051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11b5ff-6788-4bd1-ac95-db5e9f90e726 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:10.189528 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11b5ff-6788-4bd1-ac95-db5e9f90e726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:10.206828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2b11b5ff-6788-4bd1-ac95-db5e9f90e726 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:10.207557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c527ab9-202f-4f41-912a-0478342a4aef None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:10.208087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2b11b5ff-6788-4bd1-ac95-db5e9f90e726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:10.244356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7dce4-bf37-43c7-8a13-c07ebe4c28b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:10.245055 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7dce4-bf37-43c7-8a13-c07ebe4c28b2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:10.245325 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7dce4-bf37-43c7-8a13-c07ebe4c28b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:10.245610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7dce4-bf37-43c7-8a13-c07ebe4c28b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:10.246373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7dce4-bf37-43c7-8a13-c07ebe4c28b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:10.246738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7dce4-bf37-43c7-8a13-c07ebe4c28b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:10.248146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7dce4-bf37-43c7-8a13-c07ebe4c28b2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:10.257370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 23/51] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:10.262910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7dce4-bf37-43c7-8a13-c07ebe4c28b2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:10.272437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:10.274607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:10.282830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 29/52] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:10.306297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11b5ff-6788-4bd1-ac95-db5e9f90e726 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:10.372661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7dce4-bf37-43c7-8a13-c07ebe4c28b2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:10.411971 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dcdba-efa1-4e41-88a2-3b29ca40b248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:10.412889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dcdba-efa1-4e41-88a2-3b29ca40b248 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:10.413174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dcdba-efa1-4e41-88a2-3b29ca40b248 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:10.413435 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dcdba-efa1-4e41-88a2-3b29ca40b248 None admin] PATH_INFO: `/v3/users/a021026b1cd24ba98923ebdb4a56f71f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:10.414863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dcdba-efa1-4e41-88a2-3b29ca40b248 None admin] RBAC: Authorizing `identity:get_user(user_id=a021026b1cd24ba98923ebdb4a56f71f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:10.416376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dcdba-efa1-4e41-88a2-3b29ca40b248 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:10.418160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 24/53] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:10 2022] GET /identity/v3/users/a021026b1cd24ba98923ebdb4a56f71f => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:10.424004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dcdba-efa1-4e41-88a2-3b29ca40b248 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:10.509829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:10.533761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dcdba-efa1-4e41-88a2-3b29ca40b248 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:10.539622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11b5ff-6788-4bd1-ac95-db5e9f90e726 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:10.569182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6fb7f-4cb1-48d9-b5c5-9d49e7a5466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:10.569915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6fb7f-4cb1-48d9-b5c5-9d49e7a5466d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:10.570208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6fb7f-4cb1-48d9-b5c5-9d49e7a5466d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:10.570489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6fb7f-4cb1-48d9-b5c5-9d49e7a5466d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:10.571750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a6fb7f-4cb1-48d9-b5c5-9d49e7a5466d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:10.573225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a6fb7f-4cb1-48d9-b5c5-9d49e7a5466d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:10.574928 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 25/54] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:10.575055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18f36c-b6de-4020-888d-c5a956df638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:10.575872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18f36c-b6de-4020-888d-c5a956df638a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:10.576238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18f36c-b6de-4020-888d-c5a956df638a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:10.576517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18f36c-b6de-4020-888d-c5a956df638a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:10.577725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18f36c-b6de-4020-888d-c5a956df638a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:10.579241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18f36c-b6de-4020-888d-c5a956df638a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:10.580784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6fb7f-4cb1-48d9-b5c5-9d49e7a5466d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:10.581199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 30/55] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:10.587592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18f36c-b6de-4020-888d-c5a956df638a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:10.653291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:10.688820 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6fb7f-4cb1-48d9-b5c5-9d49e7a5466d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:10.692228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18f36c-b6de-4020-888d-c5a956df638a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:10.729300 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ed08d-0d75-4354-ac77-a72085dc7322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:10.730111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ed08d-0d75-4354-ac77-a72085dc7322 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:10.730434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ed08d-0d75-4354-ac77-a72085dc7322 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:10.730785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ed08d-0d75-4354-ac77-a72085dc7322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:10.731632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1d3c7-e5dd-41b4-9f12-fefb399487cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:10.731739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ed08d-0d75-4354-ac77-a72085dc7322 None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.domain.id=default, user.id=a021026b1cd24ba98923ebdb4a56f71f) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:10.732020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ed08d-0d75-4354-ac77-a72085dc7322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:10.732315 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1d3c7-e5dd-41b4-9f12-fefb399487cb None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:10.732594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1d3c7-e5dd-41b4-9f12-fefb399487cb None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:10.732865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1d3c7-e5dd-41b4-9f12-fefb399487cb None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:10.733494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ed08d-0d75-4354-ac77-a72085dc7322 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:10.733581 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1d3c7-e5dd-41b4-9f12-fefb399487cb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:10.735070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1d3c7-e5dd-41b4-9f12-fefb399487cb None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:10.740222 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 26/56] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Jun 29 09:44:10 2022] GET /identity/v3/role_assignments?user.id=a021026b1cd24ba98923ebdb4a56f71f&scope.domain.id=default&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:10.769755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 31/57] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Jun 29 09:44:10 2022] POST /identity/v3/projects => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:44:12.807241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7aa25-8031-44d8-93d8-b9fca1b4fbf2 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:12.807792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7aa25-8031-44d8-93d8-b9fca1b4fbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:12.807792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7aa25-8031-44d8-93d8-b9fca1b4fbf2 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:12.809202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 27/58] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:12.814417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdccf6d-68e9-4f5d-86d1-b20e485c6441 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:12.814848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdccf6d-68e9-4f5d-86d1-b20e485c6441 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:12.815004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdccf6d-68e9-4f5d-86d1-b20e485c6441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:12.832573 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fcdccf6d-68e9-4f5d-86d1-b20e485c6441 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:12.833790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fcdccf6d-68e9-4f5d-86d1-b20e485c6441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:12.903922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 32/59] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:12.928575 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7aa25-8031-44d8-93d8-b9fca1b4fbf2 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:12.940579 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:13.036075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7aa25-8031-44d8-93d8-b9fca1b4fbf2 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:13.077270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987ab6d-2569-4aca-96af-e2f274409664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:13.078003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987ab6d-2569-4aca-96af-e2f274409664 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:13.078281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987ab6d-2569-4aca-96af-e2f274409664 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:13.078599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987ab6d-2569-4aca-96af-e2f274409664 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:13.079817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987ab6d-2569-4aca-96af-e2f274409664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:13.081342 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987ab6d-2569-4aca-96af-e2f274409664 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:13.083121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 28/60] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:13.088936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdccf6d-68e9-4f5d-86d1-b20e485c6441 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:13.139389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:13.195461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdccf6d-68e9-4f5d-86d1-b20e485c6441 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:13.231607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204c6ff-702f-47df-b8a5-083dc8e9388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:13.232275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204c6ff-702f-47df-b8a5-083dc8e9388c None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:13.232558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204c6ff-702f-47df-b8a5-083dc8e9388c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:13.232826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204c6ff-702f-47df-b8a5-083dc8e9388c None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:13.233534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204c6ff-702f-47df-b8a5-083dc8e9388c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:13.235055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204c6ff-702f-47df-b8a5-083dc8e9388c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:13.262692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 33/61] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:44:13 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:44:16.168775 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573143-b481-45f8-ab07-b1606e4a1691 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:16.169293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573143-b481-45f8-ab07-b1606e4a1691 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:16.169390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573143-b481-45f8-ab07-b1606e4a1691 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:16.170758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 29/62] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:16.180694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1696f1-c31b-4d8a-a403-f4a8be83084b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:16.181667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1696f1-c31b-4d8a-a403-f4a8be83084b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:16.182219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1696f1-c31b-4d8a-a403-f4a8be83084b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:16.207602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-9f1696f1-c31b-4d8a-a403-f4a8be83084b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:16.209217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9f1696f1-c31b-4d8a-a403-f4a8be83084b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:16.303202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 34/63] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:16.339741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee573143-b481-45f8-ab07-b1606e4a1691 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:16.429836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:16.459324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee573143-b481-45f8-ab07-b1606e4a1691 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:16.506220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd36e29-fccd-421a-afbe-5fce270d1ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:16.507123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd36e29-fccd-421a-afbe-5fce270d1ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:16.507408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd36e29-fccd-421a-afbe-5fce270d1ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:16.507682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd36e29-fccd-421a-afbe-5fce270d1ef7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:16.523709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd36e29-fccd-421a-afbe-5fce270d1ef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-8fd36e29-fccd-421a-afbe-5fce270d1ef7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:16.532970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:16.533828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:16.534642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:16.537597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 30/64] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:16.543331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1696f1-c31b-4d8a-a403-f4a8be83084b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:16.578987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:16.653907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1696f1-c31b-4d8a-a403-f4a8be83084b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:16.655326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c6c37-ae81-484f-85c3-4d520e3ff5df None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:16.655848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c6c37-ae81-484f-85c3-4d520e3ff5df None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:16.656321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c6c37-ae81-484f-85c3-4d520e3ff5df None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:16.657932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 31/65] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:16.664119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5efd2-9dfd-4416-89b8-a0089fe6f8be None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:16.664694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5efd2-9dfd-4416-89b8-a0089fe6f8be None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:16.665164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5efd2-9dfd-4416-89b8-a0089fe6f8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:16.685412 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-64c5efd2-9dfd-4416-89b8-a0089fe6f8be None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:16.687016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-64c5efd2-9dfd-4416-89b8-a0089fe6f8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:16.693802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a87e7-890c-48bb-9b72-a8fac8636406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:16.694503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a87e7-890c-48bb-9b72-a8fac8636406 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:16.694815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a87e7-890c-48bb-9b72-a8fac8636406 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:16.695092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a87e7-890c-48bb-9b72-a8fac8636406 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:16.695862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a87e7-890c-48bb-9b72-a8fac8636406 None admin] RBAC: Adding query filter params (name=member) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:16.696172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a87e7-890c-48bb-9b72-a8fac8636406 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:16.698174 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a87e7-890c-48bb-9b72-a8fac8636406 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:16.711253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 35/66] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:16.712887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c28b6-a942-4042-b0e3-553de349da5f None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:16.713396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c28b6-a942-4042-b0e3-553de349da5f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:16.713883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c28b6-a942-4042-b0e3-553de349da5f None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:16.715394 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 36/67] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:16.720919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c28b6-a942-4042-b0e3-553de349da5f None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:16.775757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 32/68] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:16.779916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43b121-e37f-4ec9-a190-2cfc9a22df17 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:16.780387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43b121-e37f-4ec9-a190-2cfc9a22df17 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:16.780859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43b121-e37f-4ec9-a190-2cfc9a22df17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:16.812229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1c43b121-e37f-4ec9-a190-2cfc9a22df17 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:16.813886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1c43b121-e37f-4ec9-a190-2cfc9a22df17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:16.856009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:16.866579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c28b6-a942-4042-b0e3-553de349da5f None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:16.890491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:16.900216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 33/69] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:16.907898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43b121-e37f-4ec9-a190-2cfc9a22df17 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:16.910691 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9b416-f67a-4585-9a78-d8214e6938f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:16.911601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9b416-f67a-4585-9a78-d8214e6938f1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:16.911888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9b416-f67a-4585-9a78-d8214e6938f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:16.912169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9b416-f67a-4585-9a78-d8214e6938f1 None admin] PATH_INFO: `/v3/users/b91b2cc826954cfd869846d1c5f8c6ae` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:16.935980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9b416-f67a-4585-9a78-d8214e6938f1 None admin] RBAC: Authorizing `identity:get_user(user_id=b91b2cc826954cfd869846d1c5f8c6ae)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:16.938537 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9b416-f67a-4585-9a78-d8214e6938f1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:16.941359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 37/70] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:16 2022] GET /identity/v3/users/b91b2cc826954cfd869846d1c5f8c6ae => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:16.947165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9b416-f67a-4585-9a78-d8214e6938f1 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:17.052924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43b121-e37f-4ec9-a190-2cfc9a22df17 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:17.066042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9b416-f67a-4585-9a78-d8214e6938f1 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:17.099386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ab1f8-ed81-4900-8b32-ff8e53a2b46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:17.100223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ab1f8-ed81-4900-8b32-ff8e53a2b46e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.100878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ab1f8-ed81-4900-8b32-ff8e53a2b46e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.101245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ab1f8-ed81-4900-8b32-ff8e53a2b46e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.102927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd4909-baaa-4c77-ac0a-dc386a2c9cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:17.103737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd4909-baaa-4c77-ac0a-dc386a2c9cfe None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.104026 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd4909-baaa-4c77-ac0a-dc386a2c9cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.104306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd4909-baaa-4c77-ac0a-dc386a2c9cfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.113933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ab1f8-ed81-4900-8b32-ff8e53a2b46e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:17.115601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd4909-baaa-4c77-ac0a-dc386a2c9cfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-899ab1f8-ed81-4900-8b32-ff8e53a2b46e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:17.122843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:17.123667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:17.124488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-b0dd4909-baaa-4c77-ac0a-dc386a2c9cfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:17.129853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:17.130663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:17.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 38/71] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:44:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:17.134554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 34/72] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:44:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:17.134554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67f62-2be2-4188-96d4-d06b6b2e3c40 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.134554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67f62-2be2-4188-96d4-d06b6b2e3c40 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.134554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b67f62-2be2-4188-96d4-d06b6b2e3c40 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.135334 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd4909-baaa-4c77-ac0a-dc386a2c9cfe None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:17.135491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 35/73] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:17.135491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b67f62-2be2-4188-96d4-d06b6b2e3c40 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:17.148501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:17.194889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:17.237150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd4909-baaa-4c77-ac0a-dc386a2c9cfe None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:17.244060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b67f62-2be2-4188-96d4-d06b6b2e3c40 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:17.272237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cacdd4-56f7-4d39-a010-a877ea511c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:17.272906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cacdd4-56f7-4d39-a010-a877ea511c61 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.273192 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cacdd4-56f7-4d39-a010-a877ea511c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.273468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cacdd4-56f7-4d39-a010-a877ea511c61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.277322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7cb93-fa6a-4ace-82b9-341e3d60ed68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:17.278014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7cb93-fa6a-4ace-82b9-341e3d60ed68 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.278298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7cb93-fa6a-4ace-82b9-341e3d60ed68 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.278592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7cb93-fa6a-4ace-82b9-341e3d60ed68 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.279349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7cb93-fa6a-4ace-82b9-341e3d60ed68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:17.279661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7cb93-fa6a-4ace-82b9-341e3d60ed68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:17.281085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7cb93-fa6a-4ace-82b9-341e3d60ed68 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:17.290200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 36/74] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:17.291654 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e93a-e140-4071-aa10-0ccce2763c98 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.291953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e93a-e140-4071-aa10-0ccce2763c98 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.292234 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e93a-e140-4071-aa10-0ccce2763c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.310960 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4709e93a-e140-4071-aa10-0ccce2763c98 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:17.312202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4709e93a-e140-4071-aa10-0ccce2763c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:17.350211 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cacdd4-56f7-4d39-a010-a877ea511c61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:17.352145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cacdd4-56f7-4d39-a010-a877ea511c61 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:17.367493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:17.383844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 37/75] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:17.384747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709e93a-e140-4071-aa10-0ccce2763c98 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:17.448375 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:17.475967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 39/76] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:17.475967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0f2f6-ac76-4b17-8853-05657d22ae8a None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.476220 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0f2f6-ac76-4b17-8853-05657d22ae8a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.476492 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0f2f6-ac76-4b17-8853-05657d22ae8a None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.478013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 40/77] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:17.478530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0f2f6-ac76-4b17-8853-05657d22ae8a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:17.510812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709e93a-e140-4071-aa10-0ccce2763c98 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:17.550947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60f01d-65b1-40ba-9a93-af0b28a76553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:17.551651 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60f01d-65b1-40ba-9a93-af0b28a76553 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.551932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60f01d-65b1-40ba-9a93-af0b28a76553 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.552205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60f01d-65b1-40ba-9a93-af0b28a76553 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.552964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60f01d-65b1-40ba-9a93-af0b28a76553 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:17.553286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60f01d-65b1-40ba-9a93-af0b28a76553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:17.554763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60f01d-65b1-40ba-9a93-af0b28a76553 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:17.563216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 38/78] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:44:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:17.568111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60f01d-65b1-40ba-9a93-af0b28a76553 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:17.594676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0f2f6-ac76-4b17-8853-05657d22ae8a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:17.610737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:17.633154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d1a7e-bf64-433b-9887-0f064d9fea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:17.634030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d1a7e-bf64-433b-9887-0f064d9fea68 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.634307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d1a7e-bf64-433b-9887-0f064d9fea68 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.634598 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d1a7e-bf64-433b-9887-0f064d9fea68 None admin] PATH_INFO: `/v3/users/a021026b1cd24ba98923ebdb4a56f71f` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.635852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d1a7e-bf64-433b-9887-0f064d9fea68 None admin] RBAC: Authorizing `identity:get_user(user_id=a021026b1cd24ba98923ebdb4a56f71f)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:17.637337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d1a7e-bf64-433b-9887-0f064d9fea68 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:17.639049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 41/79] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:17 2022] GET /identity/v3/users/a021026b1cd24ba98923ebdb4a56f71f => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:17.639899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d1a7e-bf64-433b-9887-0f064d9fea68 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:17.690709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60f01d-65b1-40ba-9a93-af0b28a76553 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:17.701244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:17.728027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de201f8b-2fdf-4c1a-8a65-0e8c9512fc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:17.728843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de201f8b-2fdf-4c1a-8a65-0e8c9512fc9c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.729134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de201f8b-2fdf-4c1a-8a65-0e8c9512fc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.729414 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de201f8b-2fdf-4c1a-8a65-0e8c9512fc9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.741191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de201f8b-2fdf-4c1a-8a65-0e8c9512fc9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:17.748008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d1a7e-bf64-433b-9887-0f064d9fea68 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-de201f8b-2fdf-4c1a-8a65-0e8c9512fc9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:17.748965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:17.749763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:17.750537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 39/80] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:17.751447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1d8e3-1de7-4e73-ba98-9b9a217d9abb None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.751753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1d8e3-1de7-4e73-ba98-9b9a217d9abb None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.751991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1d8e3-1de7-4e73-ba98-9b9a217d9abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.769901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-34a1d8e3-1de7-4e73-ba98-9b9a217d9abb None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:17.771152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-34a1d8e3-1de7-4e73-ba98-9b9a217d9abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:17.786675 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efe672-b93f-49bb-8392-f00971df9668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:17.787562 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efe672-b93f-49bb-8392-f00971df9668 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.787828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efe672-b93f-49bb-8392-f00971df9668 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.788107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efe672-b93f-49bb-8392-f00971df9668 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.789361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efe672-b93f-49bb-8392-f00971df9668 None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:17.790872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efe672-b93f-49bb-8392-f00971df9668 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:17.792631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 42/81] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:17 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:17.793425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efe672-b93f-49bb-8392-f00971df9668 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:17.843789 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 40/82] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:17.844675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1d8e3-1de7-4e73-ba98-9b9a217d9abb None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:17.857114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:17.910988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efe672-b93f-49bb-8392-f00971df9668 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:17.947683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:17.947894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20796329-99b3-4365-a89d-52ddea608cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:17.948623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20796329-99b3-4365-a89d-52ddea608cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.948899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20796329-99b3-4365-a89d-52ddea608cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.949160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20796329-99b3-4365-a89d-52ddea608cf5 None admin] PATH_INFO: `/v3/users/b91b2cc826954cfd869846d1c5f8c6ae` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:17.950679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20796329-99b3-4365-a89d-52ddea608cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=b91b2cc826954cfd869846d1c5f8c6ae)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:17.951073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1d8e3-1de7-4e73-ba98-9b9a217d9abb None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:17.952294 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20796329-99b3-4365-a89d-52ddea608cf5 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:17.954243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 43/83] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:17 2022] GET /identity/v3/users/b91b2cc826954cfd869846d1c5f8c6ae => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:17.954744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20796329-99b3-4365-a89d-52ddea608cf5 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:17.989523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09641-3667-4ee2-a9cc-ffef35e2f810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:17.990187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09641-3667-4ee2-a9cc-ffef35e2f810 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:17.990469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09641-3667-4ee2-a9cc-ffef35e2f810 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:17.990774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09641-3667-4ee2-a9cc-ffef35e2f810 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:18.064969 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d09641-3667-4ee2-a9cc-ffef35e2f810 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:18.065478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20796329-99b3-4365-a89d-52ddea608cf5 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:18.066725 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d09641-3667-4ee2-a9cc-ffef35e2f810 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:18.083858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:18.104395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d77d13-7674-48b5-bde0-ead202446ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:18.105087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d77d13-7674-48b5-bde0-ead202446ebe None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:18.105362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d77d13-7674-48b5-bde0-ead202446ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:18.105632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d77d13-7674-48b5-bde0-ead202446ebe None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:18.106397 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d77d13-7674-48b5-bde0-ead202446ebe None admin] RBAC: Adding query filter params (name=member) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:18.106749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d77d13-7674-48b5-bde0-ead202446ebe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:18.108184 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d77d13-7674-48b5-bde0-ead202446ebe None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:18.117847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 44/84] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:18.118736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d77d13-7674-48b5-bde0-ead202446ebe None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:18.169486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:18.173353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 41/85] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:18.175494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09641-3667-4ee2-a9cc-ffef35e2f810 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:18.227859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d77d13-7674-48b5-bde0-ead202446ebe None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:18.263106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38a235-f6cb-40e6-8071-78c9a71df14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:18.263765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38a235-f6cb-40e6-8071-78c9a71df14d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:18.264051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38a235-f6cb-40e6-8071-78c9a71df14d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:18.264332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38a235-f6cb-40e6-8071-78c9a71df14d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:18.265198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38a235-f6cb-40e6-8071-78c9a71df14d None admin] RBAC: Adding query filter params (role.id=f7f18826d4b44c1284ac31e25088afbd, scope.project.id=c65d427580b743dba5cb657edb44cbbc, user.id=b91b2cc826954cfd869846d1c5f8c6ae) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:18.265509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38a235-f6cb-40e6-8071-78c9a71df14d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:18.267044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38a235-f6cb-40e6-8071-78c9a71df14d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:18.276391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 45/86] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:18 2022] GET /identity/v3/role_assignments?user.id=b91b2cc826954cfd869846d1c5f8c6ae&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:18.277294 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38a235-f6cb-40e6-8071-78c9a71df14d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:18.278593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09641-3667-4ee2-a9cc-ffef35e2f810 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:18.320610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abdad2-5215-49da-a927-e57fe3a0980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:18.321461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abdad2-5215-49da-a927-e57fe3a0980e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:18.321764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abdad2-5215-49da-a927-e57fe3a0980e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:18.322024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abdad2-5215-49da-a927-e57fe3a0980e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:18.323348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abdad2-5215-49da-a927-e57fe3a0980e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:18.324892 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abdad2-5215-49da-a927-e57fe3a0980e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:18.326705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 42/87] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:18.327545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abdad2-5215-49da-a927-e57fe3a0980e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:18.349586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:18.418837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:18.431603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38a235-f6cb-40e6-8071-78c9a71df14d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:18.465233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abdad2-5215-49da-a927-e57fe3a0980e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:18.474445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8438f-39f1-4cbf-bf5b-358d4c97c012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:18.475163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8438f-39f1-4cbf-bf5b-358d4c97c012 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:18.475453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8438f-39f1-4cbf-bf5b-358d4c97c012 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:18.476097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8438f-39f1-4cbf-bf5b-358d4c97c012 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:18.524783 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980da84-7fc3-4a0b-9457-981064686740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:18.526156 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980da84-7fc3-4a0b-9457-981064686740 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:18.526672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980da84-7fc3-4a0b-9457-981064686740 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:18.527157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980da84-7fc3-4a0b-9457-981064686740 None admin] PATH_INFO: `/v3/users/b91b2cc826954cfd869846d1c5f8c6ae` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:18.529174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980da84-7fc3-4a0b-9457-981064686740 None admin] RBAC: Authorizing `identity:get_user(user_id=b91b2cc826954cfd869846d1c5f8c6ae)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:18.531792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980da84-7fc3-4a0b-9457-981064686740 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:18.534881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 43/88] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:18 2022] GET /identity/v3/users/b91b2cc826954cfd869846d1c5f8c6ae => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:18.543100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980da84-7fc3-4a0b-9457-981064686740 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:18.547955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8438f-39f1-4cbf-bf5b-358d4c97c012 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:18.549455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8438f-39f1-4cbf-bf5b-358d4c97c012 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:18.600050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:18.660447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980da84-7fc3-4a0b-9457-981064686740 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:18.665876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 46/89] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:18.666738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8438f-39f1-4cbf-bf5b-358d4c97c012 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:18.700640 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4c66b-e004-49f3-8ff9-3278ed419dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:18.701523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4c66b-e004-49f3-8ff9-3278ed419dad None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:18.701912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4c66b-e004-49f3-8ff9-3278ed419dad None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:18.702191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4c66b-e004-49f3-8ff9-3278ed419dad None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:18.703527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4c66b-e004-49f3-8ff9-3278ed419dad None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:18.705067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4c66b-e004-49f3-8ff9-3278ed419dad None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:18.712253 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 44/90] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:18 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:18.712253 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4c66b-e004-49f3-8ff9-3278ed419dad None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:18.716852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:18.780828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8438f-39f1-4cbf-bf5b-358d4c97c012 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:18.816120 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0170b5d-f7b5-484e-a890-ad37766e2a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:18.816803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0170b5d-f7b5-484e-a890-ad37766e2a46 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:18.817096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0170b5d-f7b5-484e-a890-ad37766e2a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:18.817365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0170b5d-f7b5-484e-a890-ad37766e2a46 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:18.818103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0170b5d-f7b5-484e-a890-ad37766e2a46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:18.819591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0170b5d-f7b5-484e-a890-ad37766e2a46 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:18.826224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4c66b-e004-49f3-8ff9-3278ed419dad None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:18.837216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:18.846705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 47/91] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:44:18 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:44:18.847809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0170b5d-f7b5-484e-a890-ad37766e2a46 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:18.865321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647200a-a325-4200-8f8e-1d89239e462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:18.866001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647200a-a325-4200-8f8e-1d89239e462f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:18.866282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647200a-a325-4200-8f8e-1d89239e462f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:18.866601 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647200a-a325-4200-8f8e-1d89239e462f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:18.954615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647200a-a325-4200-8f8e-1d89239e462f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:18.955963 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647200a-a325-4200-8f8e-1d89239e462f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:18.960771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:19.019937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0170b5d-f7b5-484e-a890-ad37766e2a46 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:19.067201 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e161b00-d788-48b9-93ca-ed14cf4be0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:19.068628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e161b00-d788-48b9-93ca-ed14cf4be0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:19.069387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e161b00-d788-48b9-93ca-ed14cf4be0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:19.069818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e161b00-d788-48b9-93ca-ed14cf4be0d6 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:19.071109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e161b00-d788-48b9-93ca-ed14cf4be0d6 None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:19.072603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e161b00-d788-48b9-93ca-ed14cf4be0d6 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:19.074019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 45/92] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:19.075118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647200a-a325-4200-8f8e-1d89239e462f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:19.075529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 48/93] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:18 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:19.081646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e161b00-d788-48b9-93ca-ed14cf4be0d6 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:19.173616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:19.212607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647200a-a325-4200-8f8e-1d89239e462f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:19.216432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:19.221402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e161b00-d788-48b9-93ca-ed14cf4be0d6 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:19.259025 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bec88-b0ec-4764-a96a-e5d87aa9d9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:19.259978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bec88-b0ec-4764-a96a-e5d87aa9d9dc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:19.260268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bec88-b0ec-4764-a96a-e5d87aa9d9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:19.260545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bec88-b0ec-4764-a96a-e5d87aa9d9dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:19.261404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bec88-b0ec-4764-a96a-e5d87aa9d9dc None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=c65d427580b743dba5cb657edb44cbbc, user.id=a021026b1cd24ba98923ebdb4a56f71f) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:19.261718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bec88-b0ec-4764-a96a-e5d87aa9d9dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:19.263258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bec88-b0ec-4764-a96a-e5d87aa9d9dc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:19.264925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f367be-509d-48c5-93e8-935af768eb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:19.266368 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f367be-509d-48c5-93e8-935af768eb7c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:19.266894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f367be-509d-48c5-93e8-935af768eb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:19.267378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f367be-509d-48c5-93e8-935af768eb7c None admin] PATH_INFO: `/v3/projects/2a19879d98504a7e9a5896bb58572ceb` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:19.268962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f367be-509d-48c5-93e8-935af768eb7c None admin] RBAC: Authorizing `identity:get_project(project_id=2a19879d98504a7e9a5896bb58572ceb)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:19.269600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 46/94] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:19 2022] GET /identity/v3/role_assignments?user.id=a021026b1cd24ba98923ebdb4a56f71f&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:19.270321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bec88-b0ec-4764-a96a-e5d87aa9d9dc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:19.271436 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f367be-509d-48c5-93e8-935af768eb7c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:19.273487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 49/95] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:19 2022] GET /identity/v3/projects/2a19879d98504a7e9a5896bb58572ceb => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:19.280400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f367be-509d-48c5-93e8-935af768eb7c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:19.469855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bec88-b0ec-4764-a96a-e5d87aa9d9dc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:19.537775 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e6336-6617-4c64-8bfa-2ab845410ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:19.538479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e6336-6617-4c64-8bfa-2ab845410ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:19.538781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e6336-6617-4c64-8bfa-2ab845410ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:19.539055 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e6336-6617-4c64-8bfa-2ab845410ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:19.539914 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e6336-6617-4c64-8bfa-2ab845410ef4 None admin] RBAC: Adding query filter params (role.id=f819d0f29adf41a49fa045dc3c031527, scope.project.id=c65d427580b743dba5cb657edb44cbbc, user.id=b91b2cc826954cfd869846d1c5f8c6ae) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:19.540229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e6336-6617-4c64-8bfa-2ab845410ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:19.542390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e6336-6617-4c64-8bfa-2ab845410ef4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:19.548959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f367be-509d-48c5-93e8-935af768eb7c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:19.555570 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 47/96] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:19 2022] GET /identity/v3/role_assignments?user.id=b91b2cc826954cfd869846d1c5f8c6ae&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=f819d0f29adf41a49fa045dc3c031527 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:19.592875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac2f51-e2e6-463a-b31d-f5da41f06f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:19.593590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac2f51-e2e6-463a-b31d-f5da41f06f48 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:19.593884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac2f51-e2e6-463a-b31d-f5da41f06f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:19.594166 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac2f51-e2e6-463a-b31d-f5da41f06f48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:19.595079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac2f51-e2e6-463a-b31d-f5da41f06f48 None admin] RBAC: Adding query filter params (role.id=f7f18826d4b44c1284ac31e25088afbd, scope.project.id=2a19879d98504a7e9a5896bb58572ceb, user.id=b91b2cc826954cfd869846d1c5f8c6ae) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:19.595399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac2f51-e2e6-463a-b31d-f5da41f06f48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:19.596985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac2f51-e2e6-463a-b31d-f5da41f06f48 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:19.599228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:19.607337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 50/97] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:19 2022] GET /identity/v3/role_assignments?user.id=b91b2cc826954cfd869846d1c5f8c6ae&scope.project.id=2a19879d98504a7e9a5896bb58572ceb&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:20.650658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bf422-3178-4164-95a6-c639c3cb2aea None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:20.651721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bf422-3178-4164-95a6-c639c3cb2aea None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:20.652199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bf422-3178-4164-95a6-c639c3cb2aea None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:20.653926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 48/98] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:20.659816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001d85f-3343-4002-a462-93c71fae3437 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:20.660705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001d85f-3343-4002-a462-93c71fae3437 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:20.661202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001d85f-3343-4002-a462-93c71fae3437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:20.684208 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a001d85f-3343-4002-a462-93c71fae3437 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:20.685753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a001d85f-3343-4002-a462-93c71fae3437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:20.776570 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 51/99] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:20.812009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bf422-3178-4164-95a6-c639c3cb2aea None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:20.852247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:20.947273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bf422-3178-4164-95a6-c639c3cb2aea None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:20.992597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0289d-3840-413f-8db3-f19942d9d0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:20.993776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0289d-3840-413f-8db3-f19942d9d0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:20.994275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0289d-3840-413f-8db3-f19942d9d0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:20.994779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0289d-3840-413f-8db3-f19942d9d0e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:21.009410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0289d-3840-413f-8db3-f19942d9d0e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-9ff0289d-3840-413f-8db3-f19942d9d0e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:21.017740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:21.018868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:21.019605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:21.021951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 49/100] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:21.028247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001d85f-3343-4002-a462-93c71fae3437 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:21.148958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001d85f-3343-4002-a462-93c71fae3437 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:21.177379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:21.186140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5733a2-cd5a-481c-bdee-cd17564b1994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:21.187185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5733a2-cd5a-481c-bdee-cd17564b1994 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:21.187662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5733a2-cd5a-481c-bdee-cd17564b1994 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:21.188126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5733a2-cd5a-481c-bdee-cd17564b1994 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:21.189110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5733a2-cd5a-481c-bdee-cd17564b1994 None admin] RBAC: Adding query filter params (name=member) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:21.189607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5733a2-cd5a-481c-bdee-cd17564b1994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:21.191455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5733a2-cd5a-481c-bdee-cd17564b1994 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:21.201280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 52/101] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:21.207129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0289d-3840-413f-8db3-f19942d9d0e6 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:21.318451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:21.327409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0289d-3840-413f-8db3-f19942d9d0e6 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:21.371133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c3bb8-d22c-45d9-b380-a872a021a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:21.373023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c3bb8-d22c-45d9-b380-a872a021a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:21.373781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c3bb8-d22c-45d9-b380-a872a021a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:21.374508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c3bb8-d22c-45d9-b380-a872a021a4f4 None admin] PATH_INFO: `/v3/users/b91b2cc826954cfd869846d1c5f8c6ae` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:21.376871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c3bb8-d22c-45d9-b380-a872a021a4f4 None admin] RBAC: Authorizing `identity:get_user(user_id=b91b2cc826954cfd869846d1c5f8c6ae)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:21.379928 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c3bb8-d22c-45d9-b380-a872a021a4f4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:21.383524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 50/102] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:21 2022] GET /identity/v3/users/b91b2cc826954cfd869846d1c5f8c6ae => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:21.391860 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5733a2-cd5a-481c-bdee-cd17564b1994 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:21.508284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5733a2-cd5a-481c-bdee-cd17564b1994 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:21.548123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b691bb-96c8-44cb-b4aa-ea4e13522d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:21.549277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b691bb-96c8-44cb-b4aa-ea4e13522d26 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:21.550194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b691bb-96c8-44cb-b4aa-ea4e13522d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:21.550744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b691bb-96c8-44cb-b4aa-ea4e13522d26 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:21.552387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b691bb-96c8-44cb-b4aa-ea4e13522d26 None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:21.554318 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b691bb-96c8-44cb-b4aa-ea4e13522d26 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:21.556435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 53/103] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:21 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:21.562242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c3bb8-d22c-45d9-b380-a872a021a4f4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:21.683370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c3bb8-d22c-45d9-b380-a872a021a4f4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:21.730144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb3c38-4105-4bb9-86a0-0227c72b3be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:21.731093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb3c38-4105-4bb9-86a0-0227c72b3be9 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:21.731378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb3c38-4105-4bb9-86a0-0227c72b3be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:21.731659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb3c38-4105-4bb9-86a0-0227c72b3be9 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc/users/b91b2cc826954cfd869846d1c5f8c6ae/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:21.740086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb3c38-4105-4bb9-86a0-0227c72b3be9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c65d427580b743dba5cb657edb44cbbc, user_id=b91b2cc826954cfd869846d1c5f8c6ae, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:21.741587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb3c38-4105-4bb9-86a0-0227c72b3be9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:21.763067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 51/104] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:44:21 2022] PUT /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc/users/b91b2cc826954cfd869846d1c5f8c6ae/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:44:22.002050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b665c-0775-462e-a05f-fcd6757d74b8 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:22.003051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b665c-0775-462e-a05f-fcd6757d74b8 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:22.003593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b665c-0775-462e-a05f-fcd6757d74b8 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:22.005241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 54/105] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:22.011193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c919b-5d30-481d-8466-db738cb06274 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:22.011477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c919b-5d30-481d-8466-db738cb06274 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:22.011769 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c919b-5d30-481d-8466-db738cb06274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:22.041619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3c2c919b-5d30-481d-8466-db738cb06274 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:22.041619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3c2c919b-5d30-481d-8466-db738cb06274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:22.067817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:22.152595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 52/106] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:22.188644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b665c-0775-462e-a05f-fcd6757d74b8 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:22.219038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f1b97-785c-4bcb-92c5-5a05ce02035c None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:22.219370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f1b97-785c-4bcb-92c5-5a05ce02035c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:22.219603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f1b97-785c-4bcb-92c5-5a05ce02035c None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:22.221183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 53/107] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:44:22.232296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17846e80-560d-4091-a33b-f687b2e360bb None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:22.232296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17846e80-560d-4091-a33b-f687b2e360bb None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:22.232296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17846e80-560d-4091-a33b-f687b2e360bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:22.267682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-17846e80-560d-4091-a33b-f687b2e360bb None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:22.269380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-17846e80-560d-4091-a33b-f687b2e360bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:22.271636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:22.317371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b665c-0775-462e-a05f-fcd6757d74b8 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:22.360655 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea24ac7-2af7-46fc-b32c-a964def753cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:22.361849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24ac7-2af7-46fc-b32c-a964def753cc None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:22.362365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24ac7-2af7-46fc-b32c-a964def753cc None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:22.362835 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24ac7-2af7-46fc-b32c-a964def753cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:22.375214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea24ac7-2af7-46fc-b32c-a964def753cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-bea24ac7-2af7-46fc-b32c-a964def753cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:22.384938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:22.386126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:22.387076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:22.390586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 55/108] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:22.400536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea24ac7-2af7-46fc-b32c-a964def753cc None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:22.444357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 54/109] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:22.470428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2444ec-79c6-4c42-a7f2-52af254b1e6c None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:22.470777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2444ec-79c6-4c42-a7f2-52af254b1e6c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:22.471024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2444ec-79c6-4c42-a7f2-52af254b1e6c None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:22.472850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 55/110] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:22.479668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d02f7-d8cc-460e-9560-561ea07ecf59 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:22.480232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d02f7-d8cc-460e-9560-561ea07ecf59 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:22.480695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d02f7-d8cc-460e-9560-561ea07ecf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:22.501362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ff5d02f7-d8cc-460e-9560-561ea07ecf59 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:22.502963 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ff5d02f7-d8cc-460e-9560-561ea07ecf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:22.531433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea24ac7-2af7-46fc-b32c-a964def753cc None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:22.562979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:22.567863 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a408394-1757-4ef1-a953-e195312df4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:22.568497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408394-1757-4ef1-a953-e195312df4af None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:22.568791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408394-1757-4ef1-a953-e195312df4af None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:22.569076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408394-1757-4ef1-a953-e195312df4af None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:22.569848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a408394-1757-4ef1-a953-e195312df4af None admin] RBAC: Adding query filter params (name=reader) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:22.570165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a408394-1757-4ef1-a953-e195312df4af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:22.571722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a408394-1757-4ef1-a953-e195312df4af None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:22.582035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 56/111] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:22.587418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a408394-1757-4ef1-a953-e195312df4af None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:22.645436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 56/112] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:22.646396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d02f7-d8cc-460e-9560-561ea07ecf59 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:22.675632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:22.712235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a408394-1757-4ef1-a953-e195312df4af None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:22.753154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dfbab-41df-4ab3-aa6b-cb8d519889d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:22.753998 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dfbab-41df-4ab3-aa6b-cb8d519889d1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:22.754290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dfbab-41df-4ab3-aa6b-cb8d519889d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:22.754596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dfbab-41df-4ab3-aa6b-cb8d519889d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:22.767405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828dfbab-41df-4ab3-aa6b-cb8d519889d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-828dfbab-41df-4ab3-aa6b-cb8d519889d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:22.780792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:22.781651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:22.782401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:22.785223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 57/113] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:44:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:22.786139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dfbab-41df-4ab3-aa6b-cb8d519889d1 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:22.874885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:22.892759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d02f7-d8cc-460e-9560-561ea07ecf59 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:22.934415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfd626-e2a5-41bc-8645-d5d8332faa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:22.935515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfd626-e2a5-41bc-8645-d5d8332faa4a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:22.935816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfd626-e2a5-41bc-8645-d5d8332faa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:22.936103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfd626-e2a5-41bc-8645-d5d8332faa4a None admin] PATH_INFO: `/v3/users/bed63c29419946fd974576a61c3a9ef2` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:22.951758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfd626-e2a5-41bc-8645-d5d8332faa4a None admin] RBAC: Authorizing `identity:get_user(user_id=bed63c29419946fd974576a61c3a9ef2)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:22.953254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfd626-e2a5-41bc-8645-d5d8332faa4a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:22.965755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 57/114] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:22 2022] GET /identity/v3/users/bed63c29419946fd974576a61c3a9ef2 => generated 289 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:22.965755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfd626-e2a5-41bc-8645-d5d8332faa4a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:22.986136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dfbab-41df-4ab3-aa6b-cb8d519889d1 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:23.027390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d61c48-a1d6-45a0-90a0-1d4500f2b9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:23.028935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d61c48-a1d6-45a0-90a0-1d4500f2b9bb None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:23.029217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d61c48-a1d6-45a0-90a0-1d4500f2b9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:23.029751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d61c48-a1d6-45a0-90a0-1d4500f2b9bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:23.030909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d61c48-a1d6-45a0-90a0-1d4500f2b9bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:23.032388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d61c48-a1d6-45a0-90a0-1d4500f2b9bb None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:23.034470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 58/115] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:23.035284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:23.035451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d61c48-a1d6-45a0-90a0-1d4500f2b9bb None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:23.090869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfd626-e2a5-41bc-8645-d5d8332faa4a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:23.136287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eae01f-0772-4e03-877e-77fffc29b5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:23.136995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eae01f-0772-4e03-877e-77fffc29b5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:23.137192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eae01f-0772-4e03-877e-77fffc29b5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:23.137469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eae01f-0772-4e03-877e-77fffc29b5b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:23.138241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eae01f-0772-4e03-877e-77fffc29b5b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:23.138553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eae01f-0772-4e03-877e-77fffc29b5b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:23.139994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eae01f-0772-4e03-877e-77fffc29b5b7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:23.149296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 58/116] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:23.152012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb35c7-41d1-45ab-a82e-3e0a6baebe2b None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:23.152524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb35c7-41d1-45ab-a82e-3e0a6baebe2b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:23.152981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb35c7-41d1-45ab-a82e-3e0a6baebe2b None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:23.154649 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 59/117] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:23.156086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da917b-efff-4d27-9511-cfb9d9ad23eb None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:23.156602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da917b-efff-4d27-9511-cfb9d9ad23eb None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:23.157091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da917b-efff-4d27-9511-cfb9d9ad23eb None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:23.159210 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 60/118] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:23.160205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da917b-efff-4d27-9511-cfb9d9ad23eb None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:23.182116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d61c48-a1d6-45a0-90a0-1d4500f2b9bb None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:23.192433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:23.226805 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857c842-5333-4534-97e6-c60295c61b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:23.227502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857c842-5333-4534-97e6-c60295c61b98 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:23.227873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857c842-5333-4534-97e6-c60295c61b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:23.228149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857c842-5333-4534-97e6-c60295c61b98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:23.289809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da917b-efff-4d27-9511-cfb9d9ad23eb None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:23.315196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857c842-5333-4534-97e6-c60295c61b98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:23.316720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857c842-5333-4534-97e6-c60295c61b98 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:23.325170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:23.333453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb3bf8-326a-4215-ba91-ef122909f965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:23.334241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb3bf8-326a-4215-ba91-ef122909f965 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:23.334514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb3bf8-326a-4215-ba91-ef122909f965 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:23.334825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb3bf8-326a-4215-ba91-ef122909f965 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:23.335737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb3bf8-326a-4215-ba91-ef122909f965 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:23.338012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb3bf8-326a-4215-ba91-ef122909f965 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:23.369762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 61/119] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:44:23 2022] POST /identity/v3/projects => generated 302 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:44:23.371155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb3bf8-326a-4215-ba91-ef122909f965 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:23.488186 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 59/120] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:23.489520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1863ec1-3d21-44bb-864f-a57928b01210 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:23.489810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1863ec1-3d21-44bb-864f-a57928b01210 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:23.490090 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1863ec1-3d21-44bb-864f-a57928b01210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:23.516836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-e1863ec1-3d21-44bb-864f-a57928b01210 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:23.519159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e1863ec1-3d21-44bb-864f-a57928b01210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:23.522880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:23.533438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb3bf8-326a-4215-ba91-ef122909f965 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:23.578469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7680f3-476e-4ef9-8d81-ff997090bf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:23.580268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7680f3-476e-4ef9-8d81-ff997090bf6f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:23.580791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7680f3-476e-4ef9-8d81-ff997090bf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:23.582002 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7680f3-476e-4ef9-8d81-ff997090bf6f None admin] PATH_INFO: `/v3/users/a021026b1cd24ba98923ebdb4a56f71f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:23.583772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7680f3-476e-4ef9-8d81-ff997090bf6f None admin] RBAC: Authorizing `identity:get_user(user_id=a021026b1cd24ba98923ebdb4a56f71f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:23.586895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7680f3-476e-4ef9-8d81-ff997090bf6f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:23.594591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 62/121] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:23 2022] GET /identity/v3/users/a021026b1cd24ba98923ebdb4a56f71f => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:23.594591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef8047-e0b5-4d56-aada-42ffe78e5216 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:23.594591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef8047-e0b5-4d56-aada-42ffe78e5216 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:23.594591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef8047-e0b5-4d56-aada-42ffe78e5216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:23.613097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-cbef8047-e0b5-4d56-aada-42ffe78e5216 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:23.614312 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-cbef8047-e0b5-4d56-aada-42ffe78e5216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:23.621962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 60/122] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:23.622697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1863ec1-3d21-44bb-864f-a57928b01210 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:23.684391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:23.718248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 63/123] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:23.719327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef8047-e0b5-4d56-aada-42ffe78e5216 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:23.765834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1863ec1-3d21-44bb-864f-a57928b01210 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:23.809453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5192b3a-9de1-4b02-875b-289cf5b25f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:23.810348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5192b3a-9de1-4b02-875b-289cf5b25f27 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:23.810660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5192b3a-9de1-4b02-875b-289cf5b25f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:23.810940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5192b3a-9de1-4b02-875b-289cf5b25f27 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:23.812642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5192b3a-9de1-4b02-875b-289cf5b25f27 None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:23.814527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5192b3a-9de1-4b02-875b-289cf5b25f27 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:23.817351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 61/124] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:23 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:23.818178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5192b3a-9de1-4b02-875b-289cf5b25f27 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:23.829688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:23.842011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef8047-e0b5-4d56-aada-42ffe78e5216 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:23.892632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349155d4-6a82-4232-bc94-008ac56c7cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:23.894120 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349155d4-6a82-4232-bc94-008ac56c7cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:23.894834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349155d4-6a82-4232-bc94-008ac56c7cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:23.895032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349155d4-6a82-4232-bc94-008ac56c7cc8 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:23.896538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349155d4-6a82-4232-bc94-008ac56c7cc8 None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:23.898689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349155d4-6a82-4232-bc94-008ac56c7cc8 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:23.901362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 64/125] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:23 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:23.902272 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349155d4-6a82-4232-bc94-008ac56c7cc8 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:23.972270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:23.984973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5192b3a-9de1-4b02-875b-289cf5b25f27 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:24.032292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3c56b-20f5-49d8-9745-fd81f85f869e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:24.033123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3c56b-20f5-49d8-9745-fd81f85f869e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:24.033480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3c56b-20f5-49d8-9745-fd81f85f869e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:24.033827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3c56b-20f5-49d8-9745-fd81f85f869e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:24.058108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349155d4-6a82-4232-bc94-008ac56c7cc8 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:24.059947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3c56b-20f5-49d8-9745-fd81f85f869e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-6bc3c56b-20f5-49d8-9745-fd81f85f869e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:24.071584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:24.072965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:24.074167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:24.077766 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 62/126] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:24.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3c56b-20f5-49d8-9745-fd81f85f869e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:24.114383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035eba24-2e43-4667-b7ae-db8e42105576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:24.115635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eba24-2e43-4667-b7ae-db8e42105576 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:24.116114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eba24-2e43-4667-b7ae-db8e42105576 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:24.116803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eba24-2e43-4667-b7ae-db8e42105576 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:24.135408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035eba24-2e43-4667-b7ae-db8e42105576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-035eba24-2e43-4667-b7ae-db8e42105576 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:24.147441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:24.148327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 65/127] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:44:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:24.150075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035eba24-2e43-4667-b7ae-db8e42105576 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:24.171964 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:24.218748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3c56b-20f5-49d8-9745-fd81f85f869e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:24.255729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e67011-f838-4994-b985-15d6574eb464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:24.256512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e67011-f838-4994-b985-15d6574eb464 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:24.256845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e67011-f838-4994-b985-15d6574eb464 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:24.257123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e67011-f838-4994-b985-15d6574eb464 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:24.257977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e67011-f838-4994-b985-15d6574eb464 None admin] RBAC: Adding query filter params (role.id=61240f79e73148dbbfc847f44c528190, scope.project.id=c65d427580b743dba5cb657edb44cbbc, user.id=bed63c29419946fd974576a61c3a9ef2) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:24.258293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e67011-f838-4994-b985-15d6574eb464 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:24.259918 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e67011-f838-4994-b985-15d6574eb464 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:24.266631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 63/128] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:24 2022] GET /identity/v3/role_assignments?user.id=bed63c29419946fd974576a61c3a9ef2&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=61240f79e73148dbbfc847f44c528190 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:24.267668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e67011-f838-4994-b985-15d6574eb464 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:24.278812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035eba24-2e43-4667-b7ae-db8e42105576 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:24.299456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:24.383967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04396da5-8677-409b-b4db-709809ee44da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:24.385790 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04396da5-8677-409b-b4db-709809ee44da None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:24.385980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e67011-f838-4994-b985-15d6574eb464 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:24.386612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04396da5-8677-409b-b4db-709809ee44da None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:24.387095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04396da5-8677-409b-b4db-709809ee44da None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc/users/a021026b1cd24ba98923ebdb4a56f71f/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:24.397612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04396da5-8677-409b-b4db-709809ee44da None admin] RBAC: Authorizing `identity:create_grant(project_id=c65d427580b743dba5cb657edb44cbbc, user_id=a021026b1cd24ba98923ebdb4a56f71f, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:24.400445 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04396da5-8677-409b-b4db-709809ee44da None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:24.420879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:24.428550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6724450-8152-4b6d-a6fc-7c164d0c1846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:24.429590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6724450-8152-4b6d-a6fc-7c164d0c1846 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:24.430084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6724450-8152-4b6d-a6fc-7c164d0c1846 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:24.430512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6724450-8152-4b6d-a6fc-7c164d0c1846 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:24.432084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6724450-8152-4b6d-a6fc-7c164d0c1846 None admin] RBAC: Adding query filter params (name=member) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:24.432626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6724450-8152-4b6d-a6fc-7c164d0c1846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:24.435729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6724450-8152-4b6d-a6fc-7c164d0c1846 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:24.442276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 66/129] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:44:24 2022] PUT /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc/users/a021026b1cd24ba98923ebdb4a56f71f/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:44:24.443531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04396da5-8677-409b-b4db-709809ee44da None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:24.447511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 64/130] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:24.466331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6724450-8152-4b6d-a6fc-7c164d0c1846 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:24.656458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04396da5-8677-409b-b4db-709809ee44da None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:24.664103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6724450-8152-4b6d-a6fc-7c164d0c1846 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:24.734423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9293d9-45f1-43d3-96f7-dff6f8591c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:24.735871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9293d9-45f1-43d3-96f7-dff6f8591c17 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:24.735976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e22797-4e97-4e18-8641-20ce36908904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:24.736154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9293d9-45f1-43d3-96f7-dff6f8591c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:24.736435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9293d9-45f1-43d3-96f7-dff6f8591c17 None admin] PATH_INFO: `/v3/users/b91b2cc826954cfd869846d1c5f8c6ae` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:24.736596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e22797-4e97-4e18-8641-20ce36908904 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:24.736878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e22797-4e97-4e18-8641-20ce36908904 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:24.737225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e22797-4e97-4e18-8641-20ce36908904 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:24.737842 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9293d9-45f1-43d3-96f7-dff6f8591c17 None admin] RBAC: Authorizing `identity:get_user(user_id=b91b2cc826954cfd869846d1c5f8c6ae)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:24.738015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e22797-4e97-4e18-8641-20ce36908904 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:24.738320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e22797-4e97-4e18-8641-20ce36908904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:24.739502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9293d9-45f1-43d3-96f7-dff6f8591c17 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:24.739760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e22797-4e97-4e18-8641-20ce36908904 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:24.741378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 65/131] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:24 2022] GET /identity/v3/users/b91b2cc826954cfd869846d1c5f8c6ae => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:24.747713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9293d9-45f1-43d3-96f7-dff6f8591c17 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:24.749947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 67/132] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:44:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:24.758039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e22797-4e97-4e18-8641-20ce36908904 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:24.804277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:24.815960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:24.863718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9293d9-45f1-43d3-96f7-dff6f8591c17 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:24.899895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f908a8b-64e7-4eef-8042-a1109ac63682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:24.900881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f908a8b-64e7-4eef-8042-a1109ac63682 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:24.901176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f908a8b-64e7-4eef-8042-a1109ac63682 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:24.901559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f908a8b-64e7-4eef-8042-a1109ac63682 None admin] PATH_INFO: `/v3/projects/2a19879d98504a7e9a5896bb58572ceb` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:24.902825 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f908a8b-64e7-4eef-8042-a1109ac63682 None admin] RBAC: Authorizing `identity:get_project(project_id=2a19879d98504a7e9a5896bb58572ceb)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:24.904357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e22797-4e97-4e18-8641-20ce36908904 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:24.904797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f908a8b-64e7-4eef-8042-a1109ac63682 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:24.906390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 66/133] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:24 2022] GET /identity/v3/projects/2a19879d98504a7e9a5896bb58572ceb => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:24.915918 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f908a8b-64e7-4eef-8042-a1109ac63682 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:24.981541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06affb-2a1f-4116-9e5b-637df32d3770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:24.983473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06affb-2a1f-4116-9e5b-637df32d3770 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:24.983765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06affb-2a1f-4116-9e5b-637df32d3770 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:24.984049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06affb-2a1f-4116-9e5b-637df32d3770 None admin] PATH_INFO: `/v3/users/b91b2cc826954cfd869846d1c5f8c6ae` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:24.988618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06affb-2a1f-4116-9e5b-637df32d3770 None admin] RBAC: Authorizing `identity:get_user(user_id=b91b2cc826954cfd869846d1c5f8c6ae)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:24.990126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06affb-2a1f-4116-9e5b-637df32d3770 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:24.991887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 68/134] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:24 2022] GET /identity/v3/users/b91b2cc826954cfd869846d1c5f8c6ae => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:24.998883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef8d1b-54a4-49be-886e-13ec6a211e5f None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:25.002083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef8d1b-54a4-49be-886e-13ec6a211e5f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:25.002375 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef8d1b-54a4-49be-886e-13ec6a211e5f None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:25.003995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 69/135] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:24 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:25.004741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef8d1b-54a4-49be-886e-13ec6a211e5f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:25.143726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f908a8b-64e7-4eef-8042-a1109ac63682 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:25.199306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:25.212885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e23f09-6abc-481b-a7ac-754fbceaff2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:25.213910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e23f09-6abc-481b-a7ac-754fbceaff2f None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:25.214187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e23f09-6abc-481b-a7ac-754fbceaff2f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:25.214465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e23f09-6abc-481b-a7ac-754fbceaff2f None admin] PATH_INFO: `/v3/projects/2a19879d98504a7e9a5896bb58572ceb/users/b91b2cc826954cfd869846d1c5f8c6ae/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:25.216622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e23f09-6abc-481b-a7ac-754fbceaff2f None admin] RBAC: Authorizing `identity:create_grant(project_id=2a19879d98504a7e9a5896bb58572ceb, user_id=b91b2cc826954cfd869846d1c5f8c6ae, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:25.218235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e23f09-6abc-481b-a7ac-754fbceaff2f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:25.222457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:25.229790 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef8d1b-54a4-49be-886e-13ec6a211e5f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:25.251881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 67/136] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:44:24 2022] PUT /identity/v3/projects/2a19879d98504a7e9a5896bb58572ceb/users/b91b2cc826954cfd869846d1c5f8c6ae/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:44:25.253559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baafbe2-35b9-4ba0-9e22-de54c4b19200 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:25.253851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baafbe2-35b9-4ba0-9e22-de54c4b19200 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:25.254133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baafbe2-35b9-4ba0-9e22-de54c4b19200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:25.269989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb190f-137c-4c87-bf6e-d6b79ff44b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:25.270915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb190f-137c-4c87-bf6e-d6b79ff44b12 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:25.271199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb190f-137c-4c87-bf6e-d6b79ff44b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:25.271477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb190f-137c-4c87-bf6e-d6b79ff44b12 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:25.273031 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb190f-137c-4c87-bf6e-d6b79ff44b12 None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:25.274547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb190f-137c-4c87-bf6e-d6b79ff44b12 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:25.277813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 70/137] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:25 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:25.284973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb190f-137c-4c87-bf6e-d6b79ff44b12 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:25.316183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-0baafbe2-35b9-4ba0-9e22-de54c4b19200 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:25.319766 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-0baafbe2-35b9-4ba0-9e22-de54c4b19200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:25.482608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 68/138] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:25.531803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb190f-137c-4c87-bf6e-d6b79ff44b12 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:25.579762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a1f4e-fd7b-4280-adaf-fc05653729e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:25.580693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a1f4e-fd7b-4280-adaf-fc05653729e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:25.581032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a1f4e-fd7b-4280-adaf-fc05653729e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:25.581994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a1f4e-fd7b-4280-adaf-fc05653729e5 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc/users/b91b2cc826954cfd869846d1c5f8c6ae/roles/f819d0f29adf41a49fa045dc3c031527` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:25.588647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a1f4e-fd7b-4280-adaf-fc05653729e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c65d427580b743dba5cb657edb44cbbc, user_id=b91b2cc826954cfd869846d1c5f8c6ae, role_id=f819d0f29adf41a49fa045dc3c031527)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:25.590151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a1f4e-fd7b-4280-adaf-fc05653729e5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:25.597487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baafbe2-35b9-4ba0-9e22-de54c4b19200 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:25.625049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 71/139] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:44:25 2022] PUT /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc/users/b91b2cc826954cfd869846d1c5f8c6ae/roles/f819d0f29adf41a49fa045dc3c031527 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:44:25.748089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baafbe2-35b9-4ba0-9e22-de54c4b19200 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:25.771352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:25.795958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6af0b-c7be-4eb2-b907-d6afb457de7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:25.797350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6af0b-c7be-4eb2-b907-d6afb457de7a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:25.798241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6af0b-c7be-4eb2-b907-d6afb457de7a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:25.803422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6af0b-c7be-4eb2-b907-d6afb457de7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:25.816557 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6af0b-c7be-4eb2-b907-d6afb457de7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:25.821504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622abe95-bd9e-4305-8f7f-352cddf6df97 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:25.821789 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622abe95-bd9e-4305-8f7f-352cddf6df97 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:25.822065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622abe95-bd9e-4305-8f7f-352cddf6df97 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:25.823713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 72/140] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:44:25.829075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86502568-df9f-4a98-9bf4-adc20d858ce3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:25.829379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86502568-df9f-4a98-9bf4-adc20d858ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:25.829641 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86502568-df9f-4a98-9bf4-adc20d858ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-fbb6af0b-c7be-4eb2-b907-d6afb457de7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:25.836460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:25.837356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:25.838096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 69/141] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:25.845512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6af0b-c7be-4eb2-b907-d6afb457de7a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:25.863831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-86502568-df9f-4a98-9bf4-adc20d858ce3 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:25.865053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-86502568-df9f-4a98-9bf4-adc20d858ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:25.891047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:25.989438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 73/142] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:26.004294 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6af0b-c7be-4eb2-b907-d6afb457de7a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:26.031338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86502568-df9f-4a98-9bf4-adc20d858ce3 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:26.046943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb40583-7024-4faf-87b3-7f2b1f5c0055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:26.047702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb40583-7024-4faf-87b3-7f2b1f5c0055 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:26.047980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb40583-7024-4faf-87b3-7f2b1f5c0055 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:26.048259 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb40583-7024-4faf-87b3-7f2b1f5c0055 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:26.049022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb40583-7024-4faf-87b3-7f2b1f5c0055 None admin] RBAC: Adding query filter params (name=member) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:26.049342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb40583-7024-4faf-87b3-7f2b1f5c0055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:26.052615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb40583-7024-4faf-87b3-7f2b1f5c0055 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:26.061602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 70/143] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:26.071488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb40583-7024-4faf-87b3-7f2b1f5c0055 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:26.127280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:26.209400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb40583-7024-4faf-87b3-7f2b1f5c0055 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:26.248273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e645c3-d192-404a-9d8f-875a5e56ffed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:26.249345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e645c3-d192-404a-9d8f-875a5e56ffed None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:26.249638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e645c3-d192-404a-9d8f-875a5e56ffed None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:26.249927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e645c3-d192-404a-9d8f-875a5e56ffed None admin] PATH_INFO: `/v3/users/b91b2cc826954cfd869846d1c5f8c6ae` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:26.252963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e645c3-d192-404a-9d8f-875a5e56ffed None admin] RBAC: Authorizing `identity:get_user(user_id=b91b2cc826954cfd869846d1c5f8c6ae)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:26.254506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e645c3-d192-404a-9d8f-875a5e56ffed None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:26.256414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 71/144] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:26 2022] GET /identity/v3/users/b91b2cc826954cfd869846d1c5f8c6ae => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:26.262226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e645c3-d192-404a-9d8f-875a5e56ffed None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:26.277878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86502568-df9f-4a98-9bf4-adc20d858ce3 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:26.347584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7edc-539b-47df-a3c8-0c9a976572ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:26.349008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7edc-539b-47df-a3c8-0c9a976572ba None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:26.349473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7edc-539b-47df-a3c8-0c9a976572ba None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:26.349940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7edc-539b-47df-a3c8-0c9a976572ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:26.352349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f7edc-539b-47df-a3c8-0c9a976572ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:26.355047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f7edc-539b-47df-a3c8-0c9a976572ba None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:26.362738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 74/145] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:26.364447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7edc-539b-47df-a3c8-0c9a976572ba None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:26.373344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e645c3-d192-404a-9d8f-875a5e56ffed None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:26.384890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:26.410932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bdfea-caa0-4caf-8d7e-ce5fe3cd6fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:26.411802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bdfea-caa0-4caf-8d7e-ce5fe3cd6fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:26.412158 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bdfea-caa0-4caf-8d7e-ce5fe3cd6fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:26.412517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bdfea-caa0-4caf-8d7e-ce5fe3cd6fb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:26.424339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:26.489574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bdfea-caa0-4caf-8d7e-ce5fe3cd6fb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:26.491250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bdfea-caa0-4caf-8d7e-ce5fe3cd6fb2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:26.585162 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7edc-539b-47df-a3c8-0c9a976572ba None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:26.634763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 72/146] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:26.636166 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bdfea-caa0-4caf-8d7e-ce5fe3cd6fb2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:26.651279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcd8f5-c2d1-4a9a-a5ec-df51ce263c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:26.652855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd8f5-c2d1-4a9a-a5ec-df51ce263c82 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:26.653396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:26.654676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd8f5-c2d1-4a9a-a5ec-df51ce263c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:26.656369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd8f5-c2d1-4a9a-a5ec-df51ce263c82 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:26.657325 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bcd8f5-c2d1-4a9a-a5ec-df51ce263c82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:26.660416 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bcd8f5-c2d1-4a9a-a5ec-df51ce263c82 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:26.717088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 75/147] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:44:26 2022] POST /identity/v3/users => generated 283 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:44:26.773288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bdfea-caa0-4caf-8d7e-ce5fe3cd6fb2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:26.812169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81b1eb-b68d-4874-9d33-a18935d4b78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:26.813064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81b1eb-b68d-4874-9d33-a18935d4b78c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:26.813349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81b1eb-b68d-4874-9d33-a18935d4b78c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:26.813626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81b1eb-b68d-4874-9d33-a18935d4b78c None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:26.814927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81b1eb-b68d-4874-9d33-a18935d4b78c None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:26.816513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81b1eb-b68d-4874-9d33-a18935d4b78c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:26.818701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 73/148] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:26 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:26.825282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcd8f5-c2d1-4a9a-a5ec-df51ce263c82 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:26.936239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:26.971543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcd8f5-c2d1-4a9a-a5ec-df51ce263c82 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:27.027376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c0e49-2491-44ae-85d2-5a0b64d7464f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:27.028134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c0e49-2491-44ae-85d2-5a0b64d7464f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:27.028392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c0e49-2491-44ae-85d2-5a0b64d7464f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:27.028672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c0e49-2491-44ae-85d2-5a0b64d7464f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:27.029540 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c0e49-2491-44ae-85d2-5a0b64d7464f None admin] RBAC: Adding query filter params (role.id=f7f18826d4b44c1284ac31e25088afbd, scope.project.id=c65d427580b743dba5cb657edb44cbbc, user.id=b91b2cc826954cfd869846d1c5f8c6ae) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:27.029857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c0e49-2491-44ae-85d2-5a0b64d7464f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:27.031380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c0e49-2491-44ae-85d2-5a0b64d7464f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:27.041917 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 76/149] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:26 2022] GET /identity/v3/role_assignments?user.id=b91b2cc826954cfd869846d1c5f8c6ae&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:27.376446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed0307-4d72-4582-8424-b7313563afbf None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:27.376940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed0307-4d72-4582-8424-b7313563afbf None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:27.376940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed0307-4d72-4582-8424-b7313563afbf None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:27.378838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 74/150] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:27.384767 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661ff8a-9428-4edf-b6df-20ce02c229e4 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:27.385085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661ff8a-9428-4edf-b6df-20ce02c229e4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:27.385337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661ff8a-9428-4edf-b6df-20ce02c229e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:27.405060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a661ff8a-9428-4edf-b6df-20ce02c229e4 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:27.406322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a661ff8a-9428-4edf-b6df-20ce02c229e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:27.439032 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4dce9-d470-4c28-9e2d-4028a00864f7 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:27.439330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4dce9-d470-4c28-9e2d-4028a00864f7 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:27.439587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4dce9-d470-4c28-9e2d-4028a00864f7 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:27.442607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 75/151] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:27.448520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf8a12-18ce-4c55-81e0-7049f9c73e07 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:27.448812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf8a12-18ce-4c55-81e0-7049f9c73e07 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:27.449092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf8a12-18ce-4c55-81e0-7049f9c73e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:27.472323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b8bf8a12-18ce-4c55-81e0-7049f9c73e07 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:27.473672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b8bf8a12-18ce-4c55-81e0-7049f9c73e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:27.500610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 77/152] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:27.526258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661ff8a-9428-4edf-b6df-20ce02c229e4 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:27.559548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 76/153] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:27.582826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf8a12-18ce-4c55-81e0-7049f9c73e07 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:27.591511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:27.627315 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:27.678893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661ff8a-9428-4edf-b6df-20ce02c229e4 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:27.698911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf8a12-18ce-4c55-81e0-7049f9c73e07 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:27.732203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e278c53-c69c-46e3-be11-aa409955671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:27.733423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e278c53-c69c-46e3-be11-aa409955671b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:27.733905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e278c53-c69c-46e3-be11-aa409955671b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:27.734367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e278c53-c69c-46e3-be11-aa409955671b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:27.741456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab19f3-02ad-4222-ba4e-e72b339aec22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:27.742272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab19f3-02ad-4222-ba4e-e72b339aec22 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:27.742610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab19f3-02ad-4222-ba4e-e72b339aec22 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:27.742869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab19f3-02ad-4222-ba4e-e72b339aec22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:27.751966 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e278c53-c69c-46e3-be11-aa409955671b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:27.755831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab19f3-02ad-4222-ba4e-e72b339aec22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-9e278c53-c69c-46e3-be11-aa409955671b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:27.761132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:27.761962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:27.762777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-7eab19f3-02ad-4222-ba4e-e72b339aec22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:27.763869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:27.764640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:27.765534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 77/154] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:27.768425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 78/155] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:44:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:27.771041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab19f3-02ad-4222-ba4e-e72b339aec22 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:27.774876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e278c53-c69c-46e3-be11-aa409955671b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:27.846504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:27.886633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab19f3-02ad-4222-ba4e-e72b339aec22 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:27.907723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e278c53-c69c-46e3-be11-aa409955671b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:27.926236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe10d14-9949-461e-ba8b-926aed1902fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:27.926988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe10d14-9949-461e-ba8b-926aed1902fe None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:27.927273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe10d14-9949-461e-ba8b-926aed1902fe None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:27.927548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe10d14-9949-461e-ba8b-926aed1902fe None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:27.928317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe10d14-9949-461e-ba8b-926aed1902fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:27.928635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe10d14-9949-461e-ba8b-926aed1902fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:27.930045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe10d14-9949-461e-ba8b-926aed1902fe None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:27.939900 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 78/156] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:27.945728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe10d14-9949-461e-ba8b-926aed1902fe None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:27.953360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf511bf9-7c33-45d9-8b9a-a31a14786950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:27.954382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf511bf9-7c33-45d9-8b9a-a31a14786950 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:27.954902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf511bf9-7c33-45d9-8b9a-a31a14786950 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:27.955371 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf511bf9-7c33-45d9-8b9a-a31a14786950 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:27.956815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf511bf9-7c33-45d9-8b9a-a31a14786950 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:27.957377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf511bf9-7c33-45d9-8b9a-a31a14786950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:27.959224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf511bf9-7c33-45d9-8b9a-a31a14786950 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:27.961208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:27.973057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 79/157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:27.979659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf511bf9-7c33-45d9-8b9a-a31a14786950 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:28.064383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe10d14-9949-461e-ba8b-926aed1902fe None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:28.098785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:28.109469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8c2cd-a9db-4d7f-85ca-1588edd67e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:28.110519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8c2cd-a9db-4d7f-85ca-1588edd67e68 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:28.110963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8c2cd-a9db-4d7f-85ca-1588edd67e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:28.111337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8c2cd-a9db-4d7f-85ca-1588edd67e68 None admin] PATH_INFO: `/v3/users/bed63c29419946fd974576a61c3a9ef2` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:28.112896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8c2cd-a9db-4d7f-85ca-1588edd67e68 None admin] RBAC: Authorizing `identity:get_user(user_id=bed63c29419946fd974576a61c3a9ef2)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:28.114677 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8c2cd-a9db-4d7f-85ca-1588edd67e68 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:28.116816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 79/158] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:27 2022] GET /identity/v3/users/bed63c29419946fd974576a61c3a9ef2 => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:28.122615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8c2cd-a9db-4d7f-85ca-1588edd67e68 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:28.246389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8c2cd-a9db-4d7f-85ca-1588edd67e68 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:28.287810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f432af-b3b0-4f68-9571-86b7a5ddc01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:28.288999 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f432af-b3b0-4f68-9571-86b7a5ddc01d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:28.289403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f432af-b3b0-4f68-9571-86b7a5ddc01d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:28.289782 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f432af-b3b0-4f68-9571-86b7a5ddc01d None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:28.291358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f432af-b3b0-4f68-9571-86b7a5ddc01d None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:28.292962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f432af-b3b0-4f68-9571-86b7a5ddc01d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:28.296669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 80/159] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:28 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:28.301752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf511bf9-7c33-45d9-8b9a-a31a14786950 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:28.302653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f432af-b3b0-4f68-9571-86b7a5ddc01d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:28.376227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:28.403905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faada9e-d07f-4f99-8135-806735020e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:28.404815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faada9e-d07f-4f99-8135-806735020e1e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:28.405088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faada9e-d07f-4f99-8135-806735020e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:28.405356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faada9e-d07f-4f99-8135-806735020e1e None admin] PATH_INFO: `/v3/users/a021026b1cd24ba98923ebdb4a56f71f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:28.411481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faada9e-d07f-4f99-8135-806735020e1e None admin] RBAC: Authorizing `identity:get_user(user_id=a021026b1cd24ba98923ebdb4a56f71f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:28.413006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faada9e-d07f-4f99-8135-806735020e1e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:28.415037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f432af-b3b0-4f68-9571-86b7a5ddc01d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:28.415529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 80/160] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:27 2022] GET /identity/v3/users/a021026b1cd24ba98923ebdb4a56f71f => generated 247 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:28.423633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faada9e-d07f-4f99-8135-806735020e1e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:28.452846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaaad0-feec-4e7d-b445-db95c564d2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:28.453805 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaaad0-feec-4e7d-b445-db95c564d2ec None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:28.454094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaaad0-feec-4e7d-b445-db95c564d2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:28.454375 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaaad0-feec-4e7d-b445-db95c564d2ec None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc/users/bed63c29419946fd974576a61c3a9ef2/roles/61240f79e73148dbbfc847f44c528190` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:28.456990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaaad0-feec-4e7d-b445-db95c564d2ec None admin] RBAC: Authorizing `identity:create_grant(project_id=c65d427580b743dba5cb657edb44cbbc, user_id=bed63c29419946fd974576a61c3a9ef2, role_id=61240f79e73148dbbfc847f44c528190)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:28.458684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaaad0-feec-4e7d-b445-db95c564d2ec None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:28.481711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 81/161] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:44:28 2022] PUT /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc/users/bed63c29419946fd974576a61c3a9ef2/roles/61240f79e73148dbbfc847f44c528190 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:44:28.544123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:28.605122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faada9e-d07f-4f99-8135-806735020e1e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:28.647243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de3d64-9810-4de2-bbb1-90e190c8e64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:28.647916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de3d64-9810-4de2-bbb1-90e190c8e64e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:28.648196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de3d64-9810-4de2-bbb1-90e190c8e64e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:28.648471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de3d64-9810-4de2-bbb1-90e190c8e64e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:28.742671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de3d64-9810-4de2-bbb1-90e190c8e64e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:28.744200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de3d64-9810-4de2-bbb1-90e190c8e64e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:28.805100 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492424b-64e5-43dc-a604-854d7fb6afae None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:28.805729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492424b-64e5-43dc-a604-854d7fb6afae None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:28.806201 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492424b-64e5-43dc-a604-854d7fb6afae None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:28.808213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 82/162] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:44:28.814915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8393c-63e0-4282-9081-cf4b59cfee1c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:28.815274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8393c-63e0-4282-9081-cf4b59cfee1c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:28.815553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8393c-63e0-4282-9081-cf4b59cfee1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:28.839659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:28.854605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-9ce8393c-63e0-4282-9081-cf4b59cfee1c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:28.854605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9ce8393c-63e0-4282-9081-cf4b59cfee1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:28.953246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 83/163] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 29 09:44:28.977677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 81/164] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:28.983998 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8393c-63e0-4282-9081-cf4b59cfee1c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:29.004678 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de3d64-9810-4de2-bbb1-90e190c8e64e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:29.081936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:29.177995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de3d64-9810-4de2-bbb1-90e190c8e64e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:29.182045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8393c-63e0-4282-9081-cf4b59cfee1c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:29.227946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a8400-bda6-4aa6-a908-f3f5e441e5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:29.228824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a8400-bda6-4aa6-a908-f3f5e441e5be None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:29.229099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a8400-bda6-4aa6-a908-f3f5e441e5be None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:29.229370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a8400-bda6-4aa6-a908-f3f5e441e5be None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:29.230589 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a8400-bda6-4aa6-a908-f3f5e441e5be None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:29.234587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a8400-bda6-4aa6-a908-f3f5e441e5be None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:29.234587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 84/165] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:28 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:29.235116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffc6da-5203-44c2-a8cb-ac34ff3abb9e None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:29.235400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffc6da-5203-44c2-a8cb-ac34ff3abb9e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:29.235673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffc6da-5203-44c2-a8cb-ac34ff3abb9e None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:29.237280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 85/166] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:29.242536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffc6da-5203-44c2-a8cb-ac34ff3abb9e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:29.275320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e16e8-77cd-4d50-91b7-c7841cc7a3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:29.276546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e16e8-77cd-4d50-91b7-c7841cc7a3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:29.277036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e16e8-77cd-4d50-91b7-c7841cc7a3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:29.277502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e16e8-77cd-4d50-91b7-c7841cc7a3f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:29.307561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e16e8-77cd-4d50-91b7-c7841cc7a3f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-616e16e8-77cd-4d50-91b7-c7841cc7a3f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:29.322617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:29.323531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:29.324543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:29.327250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 82/167] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:29.328768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ac30f-38da-4ab6-902d-f597d7efc0c1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:29.329263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ac30f-38da-4ab6-902d-f597d7efc0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:29.329735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ac30f-38da-4ab6-902d-f597d7efc0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:29.358525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffc6da-5203-44c2-a8cb-ac34ff3abb9e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:29.363838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-374ac30f-38da-4ab6-902d-f597d7efc0c1 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:29.365383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-374ac30f-38da-4ab6-902d-f597d7efc0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:29.385348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:29.400363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64decb2-d08c-48ed-b7ed-159be31e82c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:29.401327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64decb2-d08c-48ed-b7ed-159be31e82c8 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:29.401742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64decb2-d08c-48ed-b7ed-159be31e82c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:29.402186 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64decb2-d08c-48ed-b7ed-159be31e82c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:29.402918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:29.403384 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64decb2-d08c-48ed-b7ed-159be31e82c8 None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=c65d427580b743dba5cb657edb44cbbc, user.id=a021026b1cd24ba98923ebdb4a56f71f) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:29.403890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64decb2-d08c-48ed-b7ed-159be31e82c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:29.405464 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64decb2-d08c-48ed-b7ed-159be31e82c8 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:29.418604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 86/168] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:29 2022] GET /identity/v3/role_assignments?user.id=a021026b1cd24ba98923ebdb4a56f71f&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:29.418604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64decb2-d08c-48ed-b7ed-159be31e82c8 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:29.544460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 83/169] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:29.612986 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ac30f-38da-4ab6-902d-f597d7efc0c1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:29.625432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64decb2-d08c-48ed-b7ed-159be31e82c8 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:29.716463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9ba34-8760-4221-abf0-0cc3db1b51f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:29.717161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9ba34-8760-4221-abf0-0cc3db1b51f5 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:29.717461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9ba34-8760-4221-abf0-0cc3db1b51f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:29.717823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9ba34-8760-4221-abf0-0cc3db1b51f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:29.718646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9ba34-8760-4221-abf0-0cc3db1b51f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:29.718959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9ba34-8760-4221-abf0-0cc3db1b51f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:29.720412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9ba34-8760-4221-abf0-0cc3db1b51f5 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:29.749681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 87/170] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:29.753952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9ba34-8760-4221-abf0-0cc3db1b51f5 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:29.818774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ac30f-38da-4ab6-902d-f597d7efc0c1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:29.858362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bff597-f329-40b0-b009-f6bcccf488db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:29.859276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bff597-f329-40b0-b009-f6bcccf488db None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:29.859561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bff597-f329-40b0-b009-f6bcccf488db None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:29.859835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bff597-f329-40b0-b009-f6bcccf488db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:29.872618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bff597-f329-40b0-b009-f6bcccf488db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-97bff597-f329-40b0-b009-f6bcccf488db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:29.881662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:29.882618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:29.883396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 84/171] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:44:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:29.890338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bff597-f329-40b0-b009-f6bcccf488db None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:29.895035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:29.922962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:30.042656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9ba34-8760-4221-abf0-0cc3db1b51f5 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:30.149133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bff597-f329-40b0-b009-f6bcccf488db None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:30.160328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d71366-c003-42b2-bbe4-c76a1cbb1a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:30.161240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d71366-c003-42b2-bbe4-c76a1cbb1a70 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:30.161522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d71366-c003-42b2-bbe4-c76a1cbb1a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:30.161798 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d71366-c003-42b2-bbe4-c76a1cbb1a70 None admin] PATH_INFO: `/v3/users/b91b2cc826954cfd869846d1c5f8c6ae` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:30.167411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d71366-c003-42b2-bbe4-c76a1cbb1a70 None admin] RBAC: Authorizing `identity:get_user(user_id=b91b2cc826954cfd869846d1c5f8c6ae)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:30.168903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d71366-c003-42b2-bbe4-c76a1cbb1a70 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:30.170972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 88/172] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:29 2022] GET /identity/v3/users/b91b2cc826954cfd869846d1c5f8c6ae => generated 275 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:30.181667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d71366-c003-42b2-bbe4-c76a1cbb1a70 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:30.224689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fc3a3-cee6-4246-b7bc-6b75348a9f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:30.225748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fc3a3-cee6-4246-b7bc-6b75348a9f2c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:30.226026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fc3a3-cee6-4246-b7bc-6b75348a9f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:30.226293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fc3a3-cee6-4246-b7bc-6b75348a9f2c None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:30.227083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fc3a3-cee6-4246-b7bc-6b75348a9f2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:30.227383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fc3a3-cee6-4246-b7bc-6b75348a9f2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:30.228795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fc3a3-cee6-4246-b7bc-6b75348a9f2c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:30.245949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 85/173] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:44:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:30.251566 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fc3a3-cee6-4246-b7bc-6b75348a9f2c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:30.463809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d71366-c003-42b2-bbe4-c76a1cbb1a70 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:30.476958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:30.506947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:30.552974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fc3a3-cee6-4246-b7bc-6b75348a9f2c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:30.556624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc880213-90eb-4185-b762-8ffd1a566b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:30.557294 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc880213-90eb-4185-b762-8ffd1a566b14 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:30.557569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc880213-90eb-4185-b762-8ffd1a566b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:30.557838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc880213-90eb-4185-b762-8ffd1a566b14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:30.652694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c544c1-7a0a-45df-a4ec-10c8e3750c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:30.653576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c544c1-7a0a-45df-a4ec-10c8e3750c72 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:30.653854 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c544c1-7a0a-45df-a4ec-10c8e3750c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:30.654119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c544c1-7a0a-45df-a4ec-10c8e3750c72 None admin] PATH_INFO: `/v3/users/b91b2cc826954cfd869846d1c5f8c6ae` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:30.659537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c544c1-7a0a-45df-a4ec-10c8e3750c72 None admin] RBAC: Authorizing `identity:get_user(user_id=b91b2cc826954cfd869846d1c5f8c6ae)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:30.661054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c544c1-7a0a-45df-a4ec-10c8e3750c72 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:30.663183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 86/174] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:30 2022] GET /identity/v3/users/b91b2cc826954cfd869846d1c5f8c6ae => generated 275 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:30.669178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c544c1-7a0a-45df-a4ec-10c8e3750c72 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:30.699154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc880213-90eb-4185-b762-8ffd1a566b14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:30.700646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc880213-90eb-4185-b762-8ffd1a566b14 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:30.825678 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c544c1-7a0a-45df-a4ec-10c8e3750c72 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:30.866715 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcce492-910b-4c93-830a-123a3573a90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:30.867756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcce492-910b-4c93-830a-123a3573a90e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:30.868233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcce492-910b-4c93-830a-123a3573a90e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:30.868729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcce492-910b-4c93-830a-123a3573a90e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:30.886932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:30.948192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcce492-910b-4c93-830a-123a3573a90e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:30.950369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcce492-910b-4c93-830a-123a3573a90e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:30.973473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 89/175] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 792 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:30.974869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f34cb-aaf0-4f54-947a-91bd3f847ab6 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:30.975153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f34cb-aaf0-4f54-947a-91bd3f847ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:30.975786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f34cb-aaf0-4f54-947a-91bd3f847ab6 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:30.977102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 90/176] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:30.989660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f34cb-aaf0-4f54-947a-91bd3f847ab6 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:31.029221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:31.134392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 87/177] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 09:44:31.139741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242d109-f242-4d77-b494-cede0c2f89a3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.140033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242d109-f242-4d77-b494-cede0c2f89a3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.140315 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242d109-f242-4d77-b494-cede0c2f89a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.180335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2242d109-f242-4d77-b494-cede0c2f89a3 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:31.185918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2242d109-f242-4d77-b494-cede0c2f89a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:31.217599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f34cb-aaf0-4f54-947a-91bd3f847ab6 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:31.262847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb716970-0c6b-4419-b25a-0a29844a747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:31.263215 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:31.264590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb716970-0c6b-4419-b25a-0a29844a747b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.265195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb716970-0c6b-4419-b25a-0a29844a747b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.265679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb716970-0c6b-4419-b25a-0a29844a747b None admin] PATH_INFO: `/v3/projects/2a19879d98504a7e9a5896bb58572ceb` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.267650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb716970-0c6b-4419-b25a-0a29844a747b None admin] RBAC: Authorizing `identity:get_project(project_id=2a19879d98504a7e9a5896bb58572ceb)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:31.269431 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb716970-0c6b-4419-b25a-0a29844a747b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:31.272074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 91/178] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:30 2022] GET /identity/v3/projects/2a19879d98504a7e9a5896bb58572ceb => generated 312 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:31.273661 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb716970-0c6b-4419-b25a-0a29844a747b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:31.294006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 88/179] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:31.295474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460dc3c-b450-41d7-a992-c64180164d6e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.295862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460dc3c-b450-41d7-a992-c64180164d6e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.296239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460dc3c-b450-41d7-a992-c64180164d6e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.297648 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 89/180] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:31.303380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bbdf8-d355-4e34-96e5-8f61a99255f8 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.303770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bbdf8-d355-4e34-96e5-8f61a99255f8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.304270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bbdf8-d355-4e34-96e5-8f61a99255f8 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.305674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 90/181] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:31.306480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bbdf8-d355-4e34-96e5-8f61a99255f8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:31.376367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:31.396064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb716970-0c6b-4419-b25a-0a29844a747b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:31.416312 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bbdf8-d355-4e34-96e5-8f61a99255f8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:31.432885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cc0d6-2d7a-47ce-9455-7bec696306fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:31.433756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cc0d6-2d7a-47ce-9455-7bec696306fa None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.434042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cc0d6-2d7a-47ce-9455-7bec696306fa None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.434316 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cc0d6-2d7a-47ce-9455-7bec696306fa None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.435588 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7cc0d6-2d7a-47ce-9455-7bec696306fa None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:31.437057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7cc0d6-2d7a-47ce-9455-7bec696306fa None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:31.438839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 92/182] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:31 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:31.438839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ab36b-6747-4365-82a5-cb92f6d14522 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.438839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ab36b-6747-4365-82a5-cb92f6d14522 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.438839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ab36b-6747-4365-82a5-cb92f6d14522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.457982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-df5ab36b-6747-4365-82a5-cb92f6d14522 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:31.459210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-df5ab36b-6747-4365-82a5-cb92f6d14522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:31.460230 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17536d33-ac1e-4e66-b5c9-cddcbb61e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:31.460910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536d33-ac1e-4e66-b5c9-cddcbb61e53f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.461194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536d33-ac1e-4e66-b5c9-cddcbb61e53f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.461470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536d33-ac1e-4e66-b5c9-cddcbb61e53f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.462322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17536d33-ac1e-4e66-b5c9-cddcbb61e53f None admin] RBAC: Adding query filter params (role.id=f7f18826d4b44c1284ac31e25088afbd, scope.project.id=2a19879d98504a7e9a5896bb58572ceb, user.id=b91b2cc826954cfd869846d1c5f8c6ae) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:31.462672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17536d33-ac1e-4e66-b5c9-cddcbb61e53f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:31.464178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17536d33-ac1e-4e66-b5c9-cddcbb61e53f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:31.471461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 91/183] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:31 2022] GET /identity/v3/role_assignments?user.id=b91b2cc826954cfd869846d1c5f8c6ae&scope.project.id=2a19879d98504a7e9a5896bb58572ceb&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:31.472870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5e10e-0ea3-4e27-8c47-2fdbec55f5c6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.473161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5e10e-0ea3-4e27-8c47-2fdbec55f5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.473432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5e10e-0ea3-4e27-8c47-2fdbec55f5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.492475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-91f5e10e-0ea3-4e27-8c47-2fdbec55f5c6 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:31.494633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-91f5e10e-0ea3-4e27-8c47-2fdbec55f5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:31.549170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:31.559733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 93/184] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:31.560714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ab36b-6747-4365-82a5-cb92f6d14522 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:31.600154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 92/185] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:31.601741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5e10e-0ea3-4e27-8c47-2fdbec55f5c6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:31.711777 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:31.737476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5e10e-0ea3-4e27-8c47-2fdbec55f5c6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:31.739921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ab36b-6747-4365-82a5-cb92f6d14522 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:31.783252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c656d0-f464-432f-befe-3a6af30976d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:31.783785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60d059-a450-439f-842b-84a39afe63de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:31.784212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c656d0-f464-432f-befe-3a6af30976d0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.784498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c656d0-f464-432f-befe-3a6af30976d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.784614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60d059-a450-439f-842b-84a39afe63de None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.784783 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c656d0-f464-432f-befe-3a6af30976d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.784874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60d059-a450-439f-842b-84a39afe63de None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.785240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60d059-a450-439f-842b-84a39afe63de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.785620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c656d0-f464-432f-befe-3a6af30976d0 None admin] RBAC: Adding query filter params (role.id=f819d0f29adf41a49fa045dc3c031527, scope.project.id=c65d427580b743dba5cb657edb44cbbc, user.id=b91b2cc826954cfd869846d1c5f8c6ae) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:31.785932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c656d0-f464-432f-befe-3a6af30976d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:31.787434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c656d0-f464-432f-befe-3a6af30976d0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:31.793826 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 93/186] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:31 2022] GET /identity/v3/role_assignments?user.id=b91b2cc826954cfd869846d1c5f8c6ae&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=f819d0f29adf41a49fa045dc3c031527 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:31.798730 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c656d0-f464-432f-befe-3a6af30976d0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:31.798891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60d059-a450-439f-842b-84a39afe63de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-cb60d059-a450-439f-842b-84a39afe63de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:31.806946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:31.807889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:31.808687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 94/187] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:44:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:31.809332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60d059-a450-439f-842b-84a39afe63de None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:31.853569 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:31.934977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c656d0-f464-432f-befe-3a6af30976d0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:31.938313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60d059-a450-439f-842b-84a39afe63de None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:31.975943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213f169-4764-4641-97d6-7f1537cb118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:31.976696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f169-4764-4641-97d6-7f1537cb118a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.976964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f169-4764-4641-97d6-7f1537cb118a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.977238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f169-4764-4641-97d6-7f1537cb118a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.978431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213f169-4764-4641-97d6-7f1537cb118a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:31.979246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3020bf6-5c7a-4734-95c4-0bc3d52801b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:31.979943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213f169-4764-4641-97d6-7f1537cb118a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:31.980064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3020bf6-5c7a-4734-95c4-0bc3d52801b6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:31.980358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3020bf6-5c7a-4734-95c4-0bc3d52801b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:31.980618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3020bf6-5c7a-4734-95c4-0bc3d52801b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:31.981581 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 94/188] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:31.982348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213f169-4764-4641-97d6-7f1537cb118a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:31.992857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3020bf6-5c7a-4734-95c4-0bc3d52801b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-b3020bf6-5c7a-4734-95c4-0bc3d52801b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:32.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:32.002353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:32.003241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 95/189] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:44:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:32.004486 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbe3c2-cc1e-4e4b-a241-278516caeff7 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:32.004486 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbe3c2-cc1e-4e4b-a241-278516caeff7 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:32.004658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbe3c2-cc1e-4e4b-a241-278516caeff7 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:32.005869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 96/190] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:32.006535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbe3c2-cc1e-4e4b-a241-278516caeff7 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:32.011411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:32.099467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:32.103009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213f169-4764-4641-97d6-7f1537cb118a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:32.125005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbe3c2-cc1e-4e4b-a241-278516caeff7 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:32.137038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148606c-5ef5-43ff-b3df-f4d8830d81f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:32.137725 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148606c-5ef5-43ff-b3df-f4d8830d81f7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:32.138004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148606c-5ef5-43ff-b3df-f4d8830d81f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:32.138278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148606c-5ef5-43ff-b3df-f4d8830d81f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:32.139065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148606c-5ef5-43ff-b3df-f4d8830d81f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:32.139370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148606c-5ef5-43ff-b3df-f4d8830d81f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:32.140781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148606c-5ef5-43ff-b3df-f4d8830d81f7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:32.149596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 95/191] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:32.150840 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32710a28-dc60-41d0-a0bb-ad30a7d8af83 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:32.151123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32710a28-dc60-41d0-a0bb-ad30a7d8af83 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:32.151394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32710a28-dc60-41d0-a0bb-ad30a7d8af83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:32.159753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d96f7f-c324-4821-a1b1-1f621f0ed9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:32.160432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d96f7f-c324-4821-a1b1-1f621f0ed9b2 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:32.160717 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d96f7f-c324-4821-a1b1-1f621f0ed9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:32.160997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d96f7f-c324-4821-a1b1-1f621f0ed9b2 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:32.161718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d96f7f-c324-4821-a1b1-1f621f0ed9b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:32.163249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d96f7f-c324-4821-a1b1-1f621f0ed9b2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:32.167849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-32710a28-dc60-41d0-a0bb-ad30a7d8af83 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:32.169034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-32710a28-dc60-41d0-a0bb-ad30a7d8af83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:32.183465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 97/192] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:44:32 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:44:32.184545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d96f7f-c324-4821-a1b1-1f621f0ed9b2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:32.254695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 96/193] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:32.260002 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32710a28-dc60-41d0-a0bb-ad30a7d8af83 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:32.266531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:32.320904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d96f7f-c324-4821-a1b1-1f621f0ed9b2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:32.366025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849e7b5-2d61-4977-9d6e-250cfe773664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:32.366755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849e7b5-2d61-4977-9d6e-250cfe773664 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:32.367038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849e7b5-2d61-4977-9d6e-250cfe773664 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:32.367314 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849e7b5-2d61-4977-9d6e-250cfe773664 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:32.368118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849e7b5-2d61-4977-9d6e-250cfe773664 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:32.368427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849e7b5-2d61-4977-9d6e-250cfe773664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:32.369819 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849e7b5-2d61-4977-9d6e-250cfe773664 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:32.373433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:32.382307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 98/194] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:44:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:32.383245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849e7b5-2d61-4977-9d6e-250cfe773664 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:32.384317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32710a28-dc60-41d0-a0bb-ad30a7d8af83 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:32.425891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d794b-5972-4d65-8d6a-268085f140d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:32.426830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d794b-5972-4d65-8d6a-268085f140d9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:32.427107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d794b-5972-4d65-8d6a-268085f140d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:32.427378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d794b-5972-4d65-8d6a-268085f140d9 None admin] PATH_INFO: `/v3/users/491e9b4bd37543c980fcb674b5e54a7b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:32.446416 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d794b-5972-4d65-8d6a-268085f140d9 None admin] RBAC: Authorizing `identity:get_user(user_id=491e9b4bd37543c980fcb674b5e54a7b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:32.447978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d794b-5972-4d65-8d6a-268085f140d9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:32.449928 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 97/195] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:32 2022] GET /identity/v3/users/491e9b4bd37543c980fcb674b5e54a7b => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:32.450544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d794b-5972-4d65-8d6a-268085f140d9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:32.519580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849e7b5-2d61-4977-9d6e-250cfe773664 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:32.547178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:32.568280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852be11-71d0-4c09-bc6d-7b1017e0afd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:32.569108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852be11-71d0-4c09-bc6d-7b1017e0afd9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:32.569384 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852be11-71d0-4c09-bc6d-7b1017e0afd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:32.569650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852be11-71d0-4c09-bc6d-7b1017e0afd9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:32.582239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852be11-71d0-4c09-bc6d-7b1017e0afd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:32.582753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d794b-5972-4d65-8d6a-268085f140d9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-c852be11-71d0-4c09-bc6d-7b1017e0afd9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:32.590126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:32.591104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:32.591884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 99/196] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:32.592533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852be11-71d0-4c09-bc6d-7b1017e0afd9 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:32.625110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34af59-9786-447b-bc56-dd120f1f23b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:32.626011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34af59-9786-447b-bc56-dd120f1f23b8 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:32.626296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34af59-9786-447b-bc56-dd120f1f23b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:32.626702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34af59-9786-447b-bc56-dd120f1f23b8 None admin] PATH_INFO: `/v3/users/491e9b4bd37543c980fcb674b5e54a7b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:32.628322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34af59-9786-447b-bc56-dd120f1f23b8 None admin] RBAC: Authorizing `identity:get_user(user_id=491e9b4bd37543c980fcb674b5e54a7b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:32.630067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34af59-9786-447b-bc56-dd120f1f23b8 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:32.632085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 98/197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:32 2022] GET /identity/v3/users/491e9b4bd37543c980fcb674b5e54a7b => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:32.632890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34af59-9786-447b-bc56-dd120f1f23b8 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:32.654946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:32.716398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852be11-71d0-4c09-bc6d-7b1017e0afd9 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:32.752671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152155e-fe47-41b3-9a89-0e173fbf8e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:32.753382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34af59-9786-447b-bc56-dd120f1f23b8 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:32.753527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152155e-fe47-41b3-9a89-0e173fbf8e7a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:32.753814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152155e-fe47-41b3-9a89-0e173fbf8e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:32.754085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152155e-fe47-41b3-9a89-0e173fbf8e7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:32.791886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ac9a9-831b-4ddb-bff2-4b1568192713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:32.792861 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ac9a9-831b-4ddb-bff2-4b1568192713 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:32.793313 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ac9a9-831b-4ddb-bff2-4b1568192713 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:32.793775 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ac9a9-831b-4ddb-bff2-4b1568192713 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:32.794746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ac9a9-831b-4ddb-bff2-4b1568192713 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:32.795231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ac9a9-831b-4ddb-bff2-4b1568192713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:32.796886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ac9a9-831b-4ddb-bff2-4b1568192713 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:32.806235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 99/198] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:32.811508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:32.812058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ac9a9-831b-4ddb-bff2-4b1568192713 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:32.839768 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152155e-fe47-41b3-9a89-0e173fbf8e7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:32.841650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152155e-fe47-41b3-9a89-0e173fbf8e7a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:32.911798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:32.939893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ac9a9-831b-4ddb-bff2-4b1568192713 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:32.970605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 100/199] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:32.970605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152155e-fe47-41b3-9a89-0e173fbf8e7a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:32.982642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e8a78-3b8b-4e72-bfc0-f941198f373d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:32.983677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e8a78-3b8b-4e72-bfc0-f941198f373d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:32.984149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e8a78-3b8b-4e72-bfc0-f941198f373d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:32.984637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e8a78-3b8b-4e72-bfc0-f941198f373d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:33.064616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e8a78-3b8b-4e72-bfc0-f941198f373d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:33.066891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e8a78-3b8b-4e72-bfc0-f941198f373d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:33.073288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:33.097056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152155e-fe47-41b3-9a89-0e173fbf8e7a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:33.135594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027fb2b-9c64-4e97-a8f8-ca516f40e5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:33.136724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027fb2b-9c64-4e97-a8f8-ca516f40e5be None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:33.137098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027fb2b-9c64-4e97-a8f8-ca516f40e5be None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:33.137676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027fb2b-9c64-4e97-a8f8-ca516f40e5be None admin] PATH_INFO: `/v3/users/bed63c29419946fd974576a61c3a9ef2` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:33.138944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027fb2b-9c64-4e97-a8f8-ca516f40e5be None admin] RBAC: Authorizing `identity:get_user(user_id=bed63c29419946fd974576a61c3a9ef2)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:33.140507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027fb2b-9c64-4e97-a8f8-ca516f40e5be None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:33.142390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 101/200] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:32 2022] GET /identity/v3/users/bed63c29419946fd974576a61c3a9ef2 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:33.147953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027fb2b-9c64-4e97-a8f8-ca516f40e5be None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:33.177523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:33.204407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 100/201] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:33.205599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e8a78-3b8b-4e72-bfc0-f941198f373d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:33.274687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027fb2b-9c64-4e97-a8f8-ca516f40e5be None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:33.312737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36350b7b-7529-4157-9cee-edd3d48b878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:33.313834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36350b7b-7529-4157-9cee-edd3d48b878f None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:33.314207 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36350b7b-7529-4157-9cee-edd3d48b878f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:33.314590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36350b7b-7529-4157-9cee-edd3d48b878f None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:33.316104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36350b7b-7529-4157-9cee-edd3d48b878f None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:33.317649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36350b7b-7529-4157-9cee-edd3d48b878f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:33.324774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 102/202] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:33 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:33.325578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e8a78-3b8b-4e72-bfc0-f941198f373d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:33.325944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36350b7b-7529-4157-9cee-edd3d48b878f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:33.345920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:33.364096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7169c92-cc3b-4b3d-b18a-0d86ccf1cdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:33.364998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7169c92-cc3b-4b3d-b18a-0d86ccf1cdaf None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:33.365411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7169c92-cc3b-4b3d-b18a-0d86ccf1cdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:33.365815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7169c92-cc3b-4b3d-b18a-0d86ccf1cdaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:33.418035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:33.439232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7169c92-cc3b-4b3d-b18a-0d86ccf1cdaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:33.441886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7169c92-cc3b-4b3d-b18a-0d86ccf1cdaf None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:33.442317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36350b7b-7529-4157-9cee-edd3d48b878f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:33.483226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804dab3-4859-4854-955b-30c83c943f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:33.484285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804dab3-4859-4854-955b-30c83c943f94 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:33.484639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804dab3-4859-4854-955b-30c83c943f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:33.485016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804dab3-4859-4854-955b-30c83c943f94 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:33.486344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804dab3-4859-4854-955b-30c83c943f94 None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:33.488953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804dab3-4859-4854-955b-30c83c943f94 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:33.491707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 103/203] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:33 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:33.498980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804dab3-4859-4854-955b-30c83c943f94 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:33.558074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 101/204] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:33.559399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7169c92-cc3b-4b3d-b18a-0d86ccf1cdaf None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:33.595288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:33.615050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804dab3-4859-4854-955b-30c83c943f94 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:33.651122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:33.652321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b217bff-6cc5-4f66-a30d-49882ae26ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:33.653214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b217bff-6cc5-4f66-a30d-49882ae26ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:33.653684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b217bff-6cc5-4f66-a30d-49882ae26ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:33.654143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b217bff-6cc5-4f66-a30d-49882ae26ac2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:33.655341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b217bff-6cc5-4f66-a30d-49882ae26ac2 None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36, user.id=491e9b4bd37543c980fcb674b5e54a7b) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:33.655913 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b217bff-6cc5-4f66-a30d-49882ae26ac2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:33.657572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b217bff-6cc5-4f66-a30d-49882ae26ac2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:33.664134 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 104/205] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:33 2022] GET /identity/v3/role_assignments?user.id=491e9b4bd37543c980fcb674b5e54a7b&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:33.671543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b217bff-6cc5-4f66-a30d-49882ae26ac2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:33.673501 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7169c92-cc3b-4b3d-b18a-0d86ccf1cdaf None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:33.722649 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02ef93-5b03-4b6a-9696-4d81488fce68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:33.726118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02ef93-5b03-4b6a-9696-4d81488fce68 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:33.726364 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02ef93-5b03-4b6a-9696-4d81488fce68 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:33.726632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02ef93-5b03-4b6a-9696-4d81488fce68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:33.727505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02ef93-5b03-4b6a-9696-4d81488fce68 None admin] RBAC: Adding query filter params (role.id=f819d0f29adf41a49fa045dc3c031527, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36, user.id=491e9b4bd37543c980fcb674b5e54a7b) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:33.727819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02ef93-5b03-4b6a-9696-4d81488fce68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:33.731796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02ef93-5b03-4b6a-9696-4d81488fce68 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:33.748383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 102/206] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:33 2022] GET /identity/v3/role_assignments?user.id=491e9b4bd37543c980fcb674b5e54a7b&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=f819d0f29adf41a49fa045dc3c031527 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:33.829550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b217bff-6cc5-4f66-a30d-49882ae26ac2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:33.889588 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacc7b7-32f5-4eb0-a3d7-023def72bd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:33.890422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacc7b7-32f5-4eb0-a3d7-023def72bd2c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:33.890738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacc7b7-32f5-4eb0-a3d7-023def72bd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:33.891018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacc7b7-32f5-4eb0-a3d7-023def72bd2c None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:33.892407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bacc7b7-32f5-4eb0-a3d7-023def72bd2c None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:33.894298 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bacc7b7-32f5-4eb0-a3d7-023def72bd2c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:33.896442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 105/207] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:33 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:33.903199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02ef93-5b03-4b6a-9696-4d81488fce68 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:34.039620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02ef93-5b03-4b6a-9696-4d81488fce68 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:34.090596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba02c5-7393-470a-bd8c-2c5d47790c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:34.091259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba02c5-7393-470a-bd8c-2c5d47790c5d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:34.091544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba02c5-7393-470a-bd8c-2c5d47790c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:34.091822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba02c5-7393-470a-bd8c-2c5d47790c5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:34.092677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba02c5-7393-470a-bd8c-2c5d47790c5d None admin] RBAC: Adding query filter params (role.id=61240f79e73148dbbfc847f44c528190, scope.project.id=c65d427580b743dba5cb657edb44cbbc, user.id=bed63c29419946fd974576a61c3a9ef2) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:34.092992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba02c5-7393-470a-bd8c-2c5d47790c5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:34.094463 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba02c5-7393-470a-bd8c-2c5d47790c5d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:34.108175 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 103/208] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:33 2022] GET /identity/v3/role_assignments?user.id=bed63c29419946fd974576a61c3a9ef2&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=61240f79e73148dbbfc847f44c528190 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:34.553873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ff967-ae79-44ba-8f24-fb04f89edcb3 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:34.554282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ff967-ae79-44ba-8f24-fb04f89edcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:34.554455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ff967-ae79-44ba-8f24-fb04f89edcb3 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:34.556011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 106/209] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:34.561498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664707dc-279d-4cc1-b90b-f281a027767e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:34.562020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664707dc-279d-4cc1-b90b-f281a027767e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:34.562509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664707dc-279d-4cc1-b90b-f281a027767e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:34.582392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-664707dc-279d-4cc1-b90b-f281a027767e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:34.583948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-664707dc-279d-4cc1-b90b-f281a027767e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:34.586425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:34.667981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 104/210] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:34.693836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ff967-ae79-44ba-8f24-fb04f89edcb3 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:34.716746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:34.755129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50b946-f8da-4fb9-aa1d-7b3012a938d1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:34.755479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50b946-f8da-4fb9-aa1d-7b3012a938d1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:34.755696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50b946-f8da-4fb9-aa1d-7b3012a938d1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:34.757281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 105/211] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:34.762954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b396752-8811-4615-9557-5406b2a8c7cf None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:34.763244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b396752-8811-4615-9557-5406b2a8c7cf None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:34.763523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b396752-8811-4615-9557-5406b2a8c7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:34.786669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6b396752-8811-4615-9557-5406b2a8c7cf None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:34.788809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6b396752-8811-4615-9557-5406b2a8c7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:34.816556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ff967-ae79-44ba-8f24-fb04f89edcb3 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:34.855078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7854d6-1606-4b52-bbda-18726674503c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:34.855884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7854d6-1606-4b52-bbda-18726674503c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:34.856170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7854d6-1606-4b52-bbda-18726674503c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:34.856446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7854d6-1606-4b52-bbda-18726674503c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:34.865546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 106/212] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:34.868637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7854d6-1606-4b52-bbda-18726674503c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-8b7854d6-1606-4b52-bbda-18726674503c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:34.882601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:34.883572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:34.884357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 107/213] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:34.886384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b396752-8811-4615-9557-5406b2a8c7cf None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:34.891421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7854d6-1606-4b52-bbda-18726674503c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:34.978173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:34.981198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:35.027753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7854d6-1606-4b52-bbda-18726674503c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:35.030290 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b396752-8811-4615-9557-5406b2a8c7cf None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:35.074258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52220fc1-9d7e-4522-b3f4-82b81c2c33d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:35.075421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52220fc1-9d7e-4522-b3f4-82b81c2c33d1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:35.075924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52220fc1-9d7e-4522-b3f4-82b81c2c33d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:35.076369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52220fc1-9d7e-4522-b3f4-82b81c2c33d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:35.077300 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52220fc1-9d7e-4522-b3f4-82b81c2c33d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:35.077813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52220fc1-9d7e-4522-b3f4-82b81c2c33d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:35.079438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52220fc1-9d7e-4522-b3f4-82b81c2c33d1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:35.089229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 107/214] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:35.100500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52220fc1-9d7e-4522-b3f4-82b81c2c33d1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:35.108900 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda48f4-d937-489a-b0c2-dfd7a0577b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:35.110165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda48f4-d937-489a-b0c2-dfd7a0577b71 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:35.110669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda48f4-d937-489a-b0c2-dfd7a0577b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:35.111142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda48f4-d937-489a-b0c2-dfd7a0577b71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:35.112781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda48f4-d937-489a-b0c2-dfd7a0577b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:35.114490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda48f4-d937-489a-b0c2-dfd7a0577b71 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:35.116502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 108/215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:35.123920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda48f4-d937-489a-b0c2-dfd7a0577b71 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:35.229775 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52220fc1-9d7e-4522-b3f4-82b81c2c33d1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:35.260983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:35.268281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac99b73-b05e-4ba5-9177-988c4f599f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:35.269572 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac99b73-b05e-4ba5-9177-988c4f599f5c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:35.270077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac99b73-b05e-4ba5-9177-988c4f599f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:35.270593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac99b73-b05e-4ba5-9177-988c4f599f5c None admin] PATH_INFO: `/v3/users/8f7cc1c0ff2847daa78400d1a1283db9` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:35.283922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda48f4-d937-489a-b0c2-dfd7a0577b71 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:35.286541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac99b73-b05e-4ba5-9177-988c4f599f5c None admin] RBAC: Authorizing `identity:get_user(user_id=8f7cc1c0ff2847daa78400d1a1283db9)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:35.288366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac99b73-b05e-4ba5-9177-988c4f599f5c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:35.290409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 108/216] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:35 2022] GET /identity/v3/users/8f7cc1c0ff2847daa78400d1a1283db9 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:35.296138 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac99b73-b05e-4ba5-9177-988c4f599f5c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:35.326169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3cb70-08d8-40d7-adbc-5356271cba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:35.327267 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3cb70-08d8-40d7-adbc-5356271cba5b None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:35.327765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3cb70-08d8-40d7-adbc-5356271cba5b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:35.328242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3cb70-08d8-40d7-adbc-5356271cba5b None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:35.329150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3cb70-08d8-40d7-adbc-5356271cba5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:35.330927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3cb70-08d8-40d7-adbc-5356271cba5b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:35.351784 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:35.357734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 109/217] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:44:35 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:44:35.418734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac99b73-b05e-4ba5-9177-988c4f599f5c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:35.458619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75b239-a6a2-42e3-896f-25a2b13144fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:35.459312 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75b239-a6a2-42e3-896f-25a2b13144fd None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:35.459599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75b239-a6a2-42e3-896f-25a2b13144fd None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:35.459883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75b239-a6a2-42e3-896f-25a2b13144fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:35.524001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:35.538146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75b239-a6a2-42e3-896f-25a2b13144fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:35.540287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75b239-a6a2-42e3-896f-25a2b13144fd None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:35.648970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 109/218] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:35.654901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3cb70-08d8-40d7-adbc-5356271cba5b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:35.779137 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3cb70-08d8-40d7-adbc-5356271cba5b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:35.815457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4471-4989-440d-81be-7a68ca88477d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:35.816036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4471-4989-440d-81be-7a68ca88477d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:35.816036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4471-4989-440d-81be-7a68ca88477d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:35.818905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 110/219] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:35.828799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e61cb-a4ec-4338-b035-c04c7833c1a6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:35.829104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e61cb-a4ec-4338-b035-c04c7833c1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:35.829420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e61cb-a4ec-4338-b035-c04c7833c1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:35.831324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba612db6-3c37-499e-809f-d285e8692441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:35.832223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba612db6-3c37-499e-809f-d285e8692441 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:35.832506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba612db6-3c37-499e-809f-d285e8692441 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:35.832786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba612db6-3c37-499e-809f-d285e8692441 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:35.834170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba612db6-3c37-499e-809f-d285e8692441 None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:35.835682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba612db6-3c37-499e-809f-d285e8692441 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:35.837467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 110/220] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:35 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:35.845897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba612db6-3c37-499e-809f-d285e8692441 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:35.871686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b37e61cb-a4ec-4338-b035-c04c7833c1a6 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:35.872915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b37e61cb-a4ec-4338-b035-c04c7833c1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:35.970996 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:36.021078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 111/221] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:36.029981 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba612db6-3c37-499e-809f-d285e8692441 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:36.064052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e61cb-a4ec-4338-b035-c04c7833c1a6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:36.093093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e082cb-a9ee-4f2f-9d6b-b07f80222d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:36.094167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e082cb-a9ee-4f2f-9d6b-b07f80222d50 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:36.094680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e082cb-a9ee-4f2f-9d6b-b07f80222d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:36.095168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e082cb-a9ee-4f2f-9d6b-b07f80222d50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:36.096384 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e082cb-a9ee-4f2f-9d6b-b07f80222d50 None admin] RBAC: Adding query filter params (role.id=f7f18826d4b44c1284ac31e25088afbd, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36, user.id=8f7cc1c0ff2847daa78400d1a1283db9) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:36.096907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e082cb-a9ee-4f2f-9d6b-b07f80222d50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:36.098689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e082cb-a9ee-4f2f-9d6b-b07f80222d50 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:36.106652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 111/222] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:35 2022] GET /identity/v3/role_assignments?user.id=8f7cc1c0ff2847daa78400d1a1283db9&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:36.108469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:36.155313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ce80-5071-4cc5-a948-3699ef0bd89b None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:36.155960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ce80-5071-4cc5-a948-3699ef0bd89b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:36.156431 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ce80-5071-4cc5-a948-3699ef0bd89b None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:36.158362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 112/223] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:36.164737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068bb15-6b20-474f-af1a-c7723d44a11f None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:36.165362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068bb15-6b20-474f-af1a-c7723d44a11f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:36.165834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068bb15-6b20-474f-af1a-c7723d44a11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:36.190007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2068bb15-6b20-474f-af1a-c7723d44a11f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:36.192423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2068bb15-6b20-474f-af1a-c7723d44a11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:36.230989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e61cb-a4ec-4338-b035-c04c7833c1a6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:36.308089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511050-fc78-432a-95bf-6d4936ca1461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:36.309408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511050-fc78-432a-95bf-6d4936ca1461 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:36.311435 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511050-fc78-432a-95bf-6d4936ca1461 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:36.311919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511050-fc78-432a-95bf-6d4936ca1461 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:36.333426 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b511050-fc78-432a-95bf-6d4936ca1461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:36.335168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 113/224] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-4b511050-fc78-432a-95bf-6d4936ca1461 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:36.341542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:36.342481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:36.343361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 112/225] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:44:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:36.349574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068bb15-6b20-474f-af1a-c7723d44a11f None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:36.370861 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511050-fc78-432a-95bf-6d4936ca1461 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:36.458279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:36.495991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068bb15-6b20-474f-af1a-c7723d44a11f None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:36.533160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1991bfb-afa9-4f5c-aaaf-06bb3a0e1515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:36.534259 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1991bfb-afa9-4f5c-aaaf-06bb3a0e1515 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:36.534767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1991bfb-afa9-4f5c-aaaf-06bb3a0e1515 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:36.535252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1991bfb-afa9-4f5c-aaaf-06bb3a0e1515 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:36.536396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1991bfb-afa9-4f5c-aaaf-06bb3a0e1515 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:36.536901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1991bfb-afa9-4f5c-aaaf-06bb3a0e1515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:36.538529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1991bfb-afa9-4f5c-aaaf-06bb3a0e1515 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:36.547905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 114/226] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:36.553829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1991bfb-afa9-4f5c-aaaf-06bb3a0e1515 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:36.584677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:36.663001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511050-fc78-432a-95bf-6d4936ca1461 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:36.710671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1991bfb-afa9-4f5c-aaaf-06bb3a0e1515 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:36.729068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3a3a5-66f8-401b-a4a3-1dcccf9b0791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:36.730253 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3a3a5-66f8-401b-a4a3-1dcccf9b0791 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:36.730772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3a3a5-66f8-401b-a4a3-1dcccf9b0791 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:36.731277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3a3a5-66f8-401b-a4a3-1dcccf9b0791 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:36.742734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3a3a5-66f8-401b-a4a3-1dcccf9b0791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-2bb3a3a5-66f8-401b-a4a3-1dcccf9b0791 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:36.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:36.753293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:36.754073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:36.756938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 113/227] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:44:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:36.763566 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3a3a5-66f8-401b-a4a3-1dcccf9b0791 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:36.766958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:36.777764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae296b6e-79c1-4780-beb8-5b466cb2f7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:36.778727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae296b6e-79c1-4780-beb8-5b466cb2f7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:36.779020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae296b6e-79c1-4780-beb8-5b466cb2f7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:36.779299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae296b6e-79c1-4780-beb8-5b466cb2f7c7 None admin] PATH_INFO: `/v3/users/491e9b4bd37543c980fcb674b5e54a7b` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:36.784012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae296b6e-79c1-4780-beb8-5b466cb2f7c7 None admin] RBAC: Authorizing `identity:get_user(user_id=491e9b4bd37543c980fcb674b5e54a7b)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:36.785440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae296b6e-79c1-4780-beb8-5b466cb2f7c7 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:36.789560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 115/228] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:36 2022] GET /identity/v3/users/491e9b4bd37543c980fcb674b5e54a7b => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:36.792775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae296b6e-79c1-4780-beb8-5b466cb2f7c7 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:36.973733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3a3a5-66f8-401b-a4a3-1dcccf9b0791 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:36.993684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae296b6e-79c1-4780-beb8-5b466cb2f7c7 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:37.044518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c54d16-a645-4aa1-a2d4-b4b52663c7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:37.045354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c54d16-a645-4aa1-a2d4-b4b52663c7bd None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:37.045637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c54d16-a645-4aa1-a2d4-b4b52663c7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:37.045907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c54d16-a645-4aa1-a2d4-b4b52663c7bd None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:37.048027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c54d16-a645-4aa1-a2d4-b4b52663c7bd None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:37.054878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c54d16-a645-4aa1-a2d4-b4b52663c7bd None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:37.057049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 116/229] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:36 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:37.063034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c54d16-a645-4aa1-a2d4-b4b52663c7bd None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:37.068041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c301b9-ef01-484f-b8a4-0989ef62986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:37.068740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c301b9-ef01-484f-b8a4-0989ef62986a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:37.069020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c301b9-ef01-484f-b8a4-0989ef62986a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:37.069294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c301b9-ef01-484f-b8a4-0989ef62986a None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:37.070063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c301b9-ef01-484f-b8a4-0989ef62986a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:37.070370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c301b9-ef01-484f-b8a4-0989ef62986a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:37.071851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c301b9-ef01-484f-b8a4-0989ef62986a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:37.086980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:37.095788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 114/230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:44:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:37.101754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c301b9-ef01-484f-b8a4-0989ef62986a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:37.153166 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:37.210820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c54d16-a645-4aa1-a2d4-b4b52663c7bd None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:37.247819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c301b9-ef01-484f-b8a4-0989ef62986a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:37.290776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8b14d-1ed4-419e-b205-254bd4e98b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:37.291691 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8b14d-1ed4-419e-b205-254bd4e98b80 None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:37.291973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8b14d-1ed4-419e-b205-254bd4e98b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:37.293160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8b14d-1ed4-419e-b205-254bd4e98b80 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/users/491e9b4bd37543c980fcb674b5e54a7b/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:37.293878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446adb80-01cd-4ce7-94f9-74337d328a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:37.294849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446adb80-01cd-4ce7-94f9-74337d328a13 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:37.295071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446adb80-01cd-4ce7-94f9-74337d328a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:37.295341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446adb80-01cd-4ce7-94f9-74337d328a13 None admin] PATH_INFO: `/v3/users/491e9b4bd37543c980fcb674b5e54a7b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:37.297910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8b14d-1ed4-419e-b205-254bd4e98b80 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea40190bfcd344248c8e0d7f7860ef36, user_id=491e9b4bd37543c980fcb674b5e54a7b, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:37.298884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446adb80-01cd-4ce7-94f9-74337d328a13 None admin] RBAC: Authorizing `identity:get_user(user_id=491e9b4bd37543c980fcb674b5e54a7b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:37.300551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8b14d-1ed4-419e-b205-254bd4e98b80 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:37.304137 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446adb80-01cd-4ce7-94f9-74337d328a13 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:37.306115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 115/231] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:37 2022] GET /identity/v3/users/491e9b4bd37543c980fcb674b5e54a7b => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:37.315996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446adb80-01cd-4ce7-94f9-74337d328a13 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:37.331670 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 117/232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:44:37 2022] PUT /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/users/491e9b4bd37543c980fcb674b5e54a7b/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:44:37.429121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:37.481244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446adb80-01cd-4ce7-94f9-74337d328a13 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:37.528078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9b310-53ff-4dca-a033-10b9c81d379b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:37.528999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9b310-53ff-4dca-a033-10b9c81d379b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:37.529283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9b310-53ff-4dca-a033-10b9c81d379b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:37.529558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9b310-53ff-4dca-a033-10b9c81d379b None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:37.535217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9b310-53ff-4dca-a033-10b9c81d379b None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:37.537064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9b310-53ff-4dca-a033-10b9c81d379b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:37.539254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 116/233] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:37 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:37.545829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8b14d-1ed4-419e-b205-254bd4e98b80 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:37.696985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8b14d-1ed4-419e-b205-254bd4e98b80 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:37.720951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:37.739568 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bb61a-a997-4ca9-8a36-08245406d0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:37.740543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bb61a-a997-4ca9-8a36-08245406d0bb None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:37.740830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bb61a-a997-4ca9-8a36-08245406d0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:37.741108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bb61a-a997-4ca9-8a36-08245406d0bb None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/users/491e9b4bd37543c980fcb674b5e54a7b/roles/f819d0f29adf41a49fa045dc3c031527` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:37.743414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bb61a-a997-4ca9-8a36-08245406d0bb None admin] RBAC: Authorizing `identity:create_grant(project_id=ea40190bfcd344248c8e0d7f7860ef36, user_id=491e9b4bd37543c980fcb674b5e54a7b, role_id=f819d0f29adf41a49fa045dc3c031527)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:37.744895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bb61a-a997-4ca9-8a36-08245406d0bb None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:37.761307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 118/234] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:44:37 2022] PUT /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/users/491e9b4bd37543c980fcb674b5e54a7b/roles/f819d0f29adf41a49fa045dc3c031527 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:44:38.013252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dca9d-110d-408a-9227-b6ee999befea None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:38.013788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dca9d-110d-408a-9227-b6ee999befea None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:38.013788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dca9d-110d-408a-9227-b6ee999befea None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:38.015447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 117/235] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:38.021122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e42f35-078f-4971-9a20-113501190242 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:38.021673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e42f35-078f-4971-9a20-113501190242 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:38.021673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e42f35-078f-4971-9a20-113501190242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:38.046912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a8e42f35-078f-4971-9a20-113501190242 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:38.048083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a8e42f35-078f-4971-9a20-113501190242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:38.123421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d6649-f2e7-4da1-a54b-866b7ae69d7c None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:38.123758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d6649-f2e7-4da1-a54b-866b7ae69d7c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:38.124009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d6649-f2e7-4da1-a54b-866b7ae69d7c None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:38.125780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 118/236] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:38.131128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3107faa-51ee-4bda-8cc9-72199b7d4d88 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:38.131412 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3107faa-51ee-4bda-8cc9-72199b7d4d88 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:38.131697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3107faa-51ee-4bda-8cc9-72199b7d4d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:38.132102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 119/237] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:38.156722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c3107faa-51ee-4bda-8cc9-72199b7d4d88 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:38.157984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c3107faa-51ee-4bda-8cc9-72199b7d4d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:38.170356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e42f35-078f-4971-9a20-113501190242 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:38.235177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 119/238] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:38.295737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3107faa-51ee-4bda-8cc9-72199b7d4d88 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:38.324959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e42f35-078f-4971-9a20-113501190242 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:38.342958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:38.367250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff64d2a-d467-416c-919b-4eb49446df13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:38.368484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff64d2a-d467-416c-919b-4eb49446df13 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:38.368960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff64d2a-d467-416c-919b-4eb49446df13 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:38.369441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff64d2a-d467-416c-919b-4eb49446df13 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:38.382615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff64d2a-d467-416c-919b-4eb49446df13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-3ff64d2a-d467-416c-919b-4eb49446df13 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:38.391710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:38.392645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:38.393560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:38.396160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 120/239] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:38.407973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff64d2a-d467-416c-919b-4eb49446df13 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:38.449700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:38.538103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff64d2a-d467-416c-919b-4eb49446df13 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:38.553647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3107faa-51ee-4bda-8cc9-72199b7d4d88 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:38.577490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309b7e-9113-41ad-b22f-dcc960483072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:38.578547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309b7e-9113-41ad-b22f-dcc960483072 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:38.579072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309b7e-9113-41ad-b22f-dcc960483072 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:38.579541 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309b7e-9113-41ad-b22f-dcc960483072 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:38.580719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309b7e-9113-41ad-b22f-dcc960483072 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:38.581328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309b7e-9113-41ad-b22f-dcc960483072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:38.583004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309b7e-9113-41ad-b22f-dcc960483072 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:38.592758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 121/240] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:38.599878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309b7e-9113-41ad-b22f-dcc960483072 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:38.635612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:38.667316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9cba8-f540-473d-8312-c27e24cf425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:38.668159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9cba8-f540-473d-8312-c27e24cf425f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:38.668443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9cba8-f540-473d-8312-c27e24cf425f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:38.668716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9cba8-f540-473d-8312-c27e24cf425f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:38.669968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9cba8-f540-473d-8312-c27e24cf425f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:38.671488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9cba8-f540-473d-8312-c27e24cf425f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:38.673510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 120/241] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:38.678930 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9cba8-f540-473d-8312-c27e24cf425f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:38.726869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309b7e-9113-41ad-b22f-dcc960483072 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:38.768273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ae8a7-00e5-45f1-a4e5-2ae125fc14c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:38.769376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ae8a7-00e5-45f1-a4e5-2ae125fc14c6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:38.769734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ae8a7-00e5-45f1-a4e5-2ae125fc14c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:38.770104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ae8a7-00e5-45f1-a4e5-2ae125fc14c6 None admin] PATH_INFO: `/v3/users/b09e64f17ad54a2289b4ec03cd0608ff` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:38.785279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ae8a7-00e5-45f1-a4e5-2ae125fc14c6 None admin] RBAC: Authorizing `identity:get_user(user_id=b09e64f17ad54a2289b4ec03cd0608ff)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:38.786987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ae8a7-00e5-45f1-a4e5-2ae125fc14c6 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:38.788910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 122/242] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:38 2022] GET /identity/v3/users/b09e64f17ad54a2289b4ec03cd0608ff => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:38.790315 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7b3f9-0d67-4a58-8eac-2ffe4318e3d0 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:38.790737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7b3f9-0d67-4a58-8eac-2ffe4318e3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:38.791108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7b3f9-0d67-4a58-8eac-2ffe4318e3d0 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:38.792498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 123/243] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:38.797572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7b3f9-0d67-4a58-8eac-2ffe4318e3d0 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:38.831780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9cba8-f540-473d-8312-c27e24cf425f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:38.868469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9322f2-79a7-4048-8fac-f04f01fb8159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:38.869306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9322f2-79a7-4048-8fac-f04f01fb8159 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:38.869683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9322f2-79a7-4048-8fac-f04f01fb8159 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:38.870055 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9322f2-79a7-4048-8fac-f04f01fb8159 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:38.870930 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9322f2-79a7-4048-8fac-f04f01fb8159 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:38.872550 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9322f2-79a7-4048-8fac-f04f01fb8159 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:38.879968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:38.899355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 121/244] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:44:38 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:44:38.900480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90af691-0ce9-43bb-8eb0-164e9266baf2 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:38.900644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90af691-0ce9-43bb-8eb0-164e9266baf2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:38.901058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90af691-0ce9-43bb-8eb0-164e9266baf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:38.923906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e90af691-0ce9-43bb-8eb0-164e9266baf2 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:38.926921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e90af691-0ce9-43bb-8eb0-164e9266baf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:38.929883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:38.944470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7b3f9-0d67-4a58-8eac-2ffe4318e3d0 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:39.001191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65673a-e7a2-4bef-946f-14e6ce90961a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:39.002175 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65673a-e7a2-4bef-946f-14e6ce90961a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:39.002773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65673a-e7a2-4bef-946f-14e6ce90961a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:39.003843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65673a-e7a2-4bef-946f-14e6ce90961a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:39.067562 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 122/245] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:39.093447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90af691-0ce9-43bb-8eb0-164e9266baf2 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:39.150872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65673a-e7a2-4bef-946f-14e6ce90961a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:39.152363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65673a-e7a2-4bef-946f-14e6ce90961a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:39.227710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90af691-0ce9-43bb-8eb0-164e9266baf2 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:39.249513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:39.262398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:39.267159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11517374-d1c8-4e0e-bdf4-cce08ad8194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:39.268110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11517374-d1c8-4e0e-bdf4-cce08ad8194f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:39.268559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11517374-d1c8-4e0e-bdf4-cce08ad8194f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:39.268843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11517374-d1c8-4e0e-bdf4-cce08ad8194f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:39.278613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 124/246] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:39.281179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11517374-d1c8-4e0e-bdf4-cce08ad8194f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:39.284525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65673a-e7a2-4bef-946f-14e6ce90961a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-11517374-d1c8-4e0e-bdf4-cce08ad8194f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:39.289630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:39.290509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:39.291689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:39.292335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 123/247] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:39.297839 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11517374-d1c8-4e0e-bdf4-cce08ad8194f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:39.416802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65673a-e7a2-4bef-946f-14e6ce90961a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:39.428131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11517374-d1c8-4e0e-bdf4-cce08ad8194f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:39.469911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34821-a46e-4033-ad44-8291791e82ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:39.471196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34821-a46e-4033-ad44-8291791e82ca None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:39.471610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34821-a46e-4033-ad44-8291791e82ca None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:39.472015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34821-a46e-4033-ad44-8291791e82ca None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:39.473688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd34821-a46e-4033-ad44-8291791e82ca None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:39.475656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ab9c3-de82-4cfe-9840-2fecf3618dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:39.475963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd34821-a46e-4033-ad44-8291791e82ca None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:39.476344 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ab9c3-de82-4cfe-9840-2fecf3618dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:39.476626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ab9c3-de82-4cfe-9840-2fecf3618dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:39.476900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ab9c3-de82-4cfe-9840-2fecf3618dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:39.477672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ab9c3-de82-4cfe-9840-2fecf3618dc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:39.477983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ab9c3-de82-4cfe-9840-2fecf3618dc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:39.478944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 125/248] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:39 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:39.479455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ab9c3-de82-4cfe-9840-2fecf3618dc1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:39.487288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34821-a46e-4033-ad44-8291791e82ca None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:39.491205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 124/249] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:39.496882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ab9c3-de82-4cfe-9840-2fecf3618dc1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:39.544852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:39.594477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:39.623730 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ab9c3-de82-4cfe-9840-2fecf3618dc1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:39.649575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34821-a46e-4033-ad44-8291791e82ca None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:39.665363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2205f2-7621-42c5-a5e9-293bcd45e245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:39.666703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2205f2-7621-42c5-a5e9-293bcd45e245 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:39.667279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2205f2-7621-42c5-a5e9-293bcd45e245 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:39.667749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2205f2-7621-42c5-a5e9-293bcd45e245 None admin] PATH_INFO: `/v3/users/8f7cc1c0ff2847daa78400d1a1283db9` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:39.669202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2205f2-7621-42c5-a5e9-293bcd45e245 None admin] RBAC: Authorizing `identity:get_user(user_id=8f7cc1c0ff2847daa78400d1a1283db9)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:39.671929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2205f2-7621-42c5-a5e9-293bcd45e245 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:39.674600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 125/250] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:39 2022] GET /identity/v3/users/8f7cc1c0ff2847daa78400d1a1283db9 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:39.680497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2205f2-7621-42c5-a5e9-293bcd45e245 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:39.691088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca481e6-fddb-432a-a5d6-da435550ee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:39.691775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca481e6-fddb-432a-a5d6-da435550ee6d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:39.692056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca481e6-fddb-432a-a5d6-da435550ee6d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:39.692329 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca481e6-fddb-432a-a5d6-da435550ee6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:39.693178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca481e6-fddb-432a-a5d6-da435550ee6d None admin] RBAC: Adding query filter params (role.id=61240f79e73148dbbfc847f44c528190, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36, user.id=b09e64f17ad54a2289b4ec03cd0608ff) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:39.693489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca481e6-fddb-432a-a5d6-da435550ee6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:39.696590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca481e6-fddb-432a-a5d6-da435550ee6d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:39.703172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 126/251] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:39 2022] GET /identity/v3/role_assignments?user.id=b09e64f17ad54a2289b4ec03cd0608ff&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=61240f79e73148dbbfc847f44c528190 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:39.811563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2205f2-7621-42c5-a5e9-293bcd45e245 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:39.824372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:39.853075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b58a1-f5ef-4254-8cdf-13e3ab1fe340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:39.854356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b58a1-f5ef-4254-8cdf-13e3ab1fe340 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:39.854938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b58a1-f5ef-4254-8cdf-13e3ab1fe340 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:39.855384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b58a1-f5ef-4254-8cdf-13e3ab1fe340 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:39.856795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b58a1-f5ef-4254-8cdf-13e3ab1fe340 None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:39.858459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b58a1-f5ef-4254-8cdf-13e3ab1fe340 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:39.860518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 126/252] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:39 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:39.867100 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca481e6-fddb-432a-a5d6-da435550ee6d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:40.023871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca481e6-fddb-432a-a5d6-da435550ee6d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:40.060895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:40.081624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bef74a-a095-4178-affa-1c94c69cfe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:40.082631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bef74a-a095-4178-affa-1c94c69cfe6f None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:40.082921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bef74a-a095-4178-affa-1c94c69cfe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:40.083200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bef74a-a095-4178-affa-1c94c69cfe6f None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/users/8f7cc1c0ff2847daa78400d1a1283db9/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:40.085347 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bef74a-a095-4178-affa-1c94c69cfe6f None admin] RBAC: Authorizing `identity:create_grant(project_id=ea40190bfcd344248c8e0d7f7860ef36, user_id=8f7cc1c0ff2847daa78400d1a1283db9, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:40.086864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bef74a-a095-4178-affa-1c94c69cfe6f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:40.109788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 127/253] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:44:39 2022] PUT /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/users/8f7cc1c0ff2847daa78400d1a1283db9/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:44:40.150100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0ea94-b065-4de7-88f9-4af61e7d6950 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:40.150649 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0ea94-b065-4de7-88f9-4af61e7d6950 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:40.150724 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0ea94-b065-4de7-88f9-4af61e7d6950 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:40.152457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 127/254] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:40.158281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73936c-df75-4361-a238-ff70c6827960 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:40.159216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73936c-df75-4361-a238-ff70c6827960 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:40.159216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73936c-df75-4361-a238-ff70c6827960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:40.177095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-cd73936c-df75-4361-a238-ff70c6827960 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:40.178626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cd73936c-df75-4361-a238-ff70c6827960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:40.263863 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 128/255] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:40.289615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0ea94-b065-4de7-88f9-4af61e7d6950 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:40.411212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0ea94-b065-4de7-88f9-4af61e7d6950 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:40.465275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857df781-0f6b-4808-883a-8fbe893090c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:40.466515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df781-0f6b-4808-883a-8fbe893090c9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:40.467027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df781-0f6b-4808-883a-8fbe893090c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:40.467530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df781-0f6b-4808-883a-8fbe893090c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:40.488117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857df781-0f6b-4808-883a-8fbe893090c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-857df781-0f6b-4808-883a-8fbe893090c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:40.496384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:40.497286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:40.498050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:40.501062 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 128/256] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:44:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:40.508006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73936c-df75-4361-a238-ff70c6827960 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:40.617997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8708b9-553e-4766-ab78-a17574231bb2 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:40.618717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8708b9-553e-4766-ab78-a17574231bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:40.619189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8708b9-553e-4766-ab78-a17574231bb2 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:40.620850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 129/257] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:44:40.627107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae963b9-b2e6-4a25-8bd0-628c7cba0974 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:40.628466 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae963b9-b2e6-4a25-8bd0-628c7cba0974 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:40.628938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae963b9-b2e6-4a25-8bd0-628c7cba0974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:40.638915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73936c-df75-4361-a238-ff70c6827960 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:40.654315 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-bae963b9-b2e6-4a25-8bd0-628c7cba0974 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:40.655965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bae963b9-b2e6-4a25-8bd0-628c7cba0974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:40.670911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:40.713324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1085cf-ff8f-461c-a14b-1644a91c0b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:40.714361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1085cf-ff8f-461c-a14b-1644a91c0b7d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:40.714907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1085cf-ff8f-461c-a14b-1644a91c0b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:40.715359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1085cf-ff8f-461c-a14b-1644a91c0b7d None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:40.716271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1085cf-ff8f-461c-a14b-1644a91c0b7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:40.716774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1085cf-ff8f-461c-a14b-1644a91c0b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:40.718552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1085cf-ff8f-461c-a14b-1644a91c0b7d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:40.727308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:40.735578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 129/258] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:40.747386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1085cf-ff8f-461c-a14b-1644a91c0b7d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:40.802047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 130/259] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:40.847459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae963b9-b2e6-4a25-8bd0-628c7cba0974 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:40.943454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1085cf-ff8f-461c-a14b-1644a91c0b7d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:41.020608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a498cd-8325-440f-8450-787f85fbcb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:41.021514 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a498cd-8325-440f-8450-787f85fbcb99 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:41.021794 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a498cd-8325-440f-8450-787f85fbcb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:41.022069 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a498cd-8325-440f-8450-787f85fbcb99 None admin] PATH_INFO: `/v3/users/491e9b4bd37543c980fcb674b5e54a7b` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:41.024313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a498cd-8325-440f-8450-787f85fbcb99 None admin] RBAC: Authorizing `identity:get_user(user_id=491e9b4bd37543c980fcb674b5e54a7b)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:41.025734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a498cd-8325-440f-8450-787f85fbcb99 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:41.027461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 130/260] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:40 2022] GET /identity/v3/users/491e9b4bd37543c980fcb674b5e54a7b => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:41.039862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a498cd-8325-440f-8450-787f85fbcb99 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:41.064273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae963b9-b2e6-4a25-8bd0-628c7cba0974 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:41.123221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:41.151336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa9ee6-3b4c-4874-a790-013d56a7f731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:41.152356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa9ee6-3b4c-4874-a790-013d56a7f731 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:41.152726 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa9ee6-3b4c-4874-a790-013d56a7f731 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:41.153118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa9ee6-3b4c-4874-a790-013d56a7f731 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:41.187233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa9ee6-3b4c-4874-a790-013d56a7f731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:41.193293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a498cd-8325-440f-8450-787f85fbcb99 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-6afa9ee6-3b4c-4874-a790-013d56a7f731 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:41.195864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:41.196752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:41.197620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:41.204458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 131/261] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:44:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:41.227806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa9ee6-3b4c-4874-a790-013d56a7f731 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:41.242322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a172d1a-09ff-4035-81d2-cfbf2e11ddac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:41.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a172d1a-09ff-4035-81d2-cfbf2e11ddac None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:41.243713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a172d1a-09ff-4035-81d2-cfbf2e11ddac None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:41.244849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a172d1a-09ff-4035-81d2-cfbf2e11ddac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:41.316047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:41.329211 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a172d1a-09ff-4035-81d2-cfbf2e11ddac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:41.331279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a172d1a-09ff-4035-81d2-cfbf2e11ddac None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:41.380719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:41.449723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 131/262] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:41.455567 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a172d1a-09ff-4035-81d2-cfbf2e11ddac None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:41.516860 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa9ee6-3b4c-4874-a790-013d56a7f731 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:41.570873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a172d1a-09ff-4035-81d2-cfbf2e11ddac None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:41.610935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f1dbb-3615-4409-8169-eae92bf4bd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:41.612105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f1dbb-3615-4409-8169-eae92bf4bd59 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:41.612569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f1dbb-3615-4409-8169-eae92bf4bd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:41.613007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f1dbb-3615-4409-8169-eae92bf4bd59 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:41.614497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f1dbb-3615-4409-8169-eae92bf4bd59 None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:41.616430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f1dbb-3615-4409-8169-eae92bf4bd59 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:41.622587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc04cc7-e558-4508-8464-9ad1c1defc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:41.622587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc04cc7-e558-4508-8464-9ad1c1defc3b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:41.622587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc04cc7-e558-4508-8464-9ad1c1defc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:41.622587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc04cc7-e558-4508-8464-9ad1c1defc3b None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:41.622984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc04cc7-e558-4508-8464-9ad1c1defc3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:41.622984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc04cc7-e558-4508-8464-9ad1c1defc3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:41.622984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc04cc7-e558-4508-8464-9ad1c1defc3b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:41.623545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 132/263] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:41 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:41.627929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f1dbb-3615-4409-8169-eae92bf4bd59 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:41.643350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 132/264] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:44:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:41.649128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc04cc7-e558-4508-8464-9ad1c1defc3b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:41.656389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:41.801998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:41.819235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc04cc7-e558-4508-8464-9ad1c1defc3b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:41.881323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa191122-2ad9-430d-bdf0-a7353ad9d23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:41.882212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa191122-2ad9-430d-bdf0-a7353ad9d23b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:41.882488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa191122-2ad9-430d-bdf0-a7353ad9d23b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:41.882782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa191122-2ad9-430d-bdf0-a7353ad9d23b None admin] PATH_INFO: `/v3/users/491e9b4bd37543c980fcb674b5e54a7b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:41.884703 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f1dbb-3615-4409-8169-eae92bf4bd59 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:41.886890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa191122-2ad9-430d-bdf0-a7353ad9d23b None admin] RBAC: Authorizing `identity:get_user(user_id=491e9b4bd37543c980fcb674b5e54a7b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:41.888381 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa191122-2ad9-430d-bdf0-a7353ad9d23b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:41.890377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 133/265] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:41 2022] GET /identity/v3/users/491e9b4bd37543c980fcb674b5e54a7b => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:41.896178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa191122-2ad9-430d-bdf0-a7353ad9d23b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:41.942400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6723b1a-7dfe-45b6-be48-1a932e03758c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:41.943143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6723b1a-7dfe-45b6-be48-1a932e03758c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:41.943432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6723b1a-7dfe-45b6-be48-1a932e03758c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:41.943709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6723b1a-7dfe-45b6-be48-1a932e03758c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:41.944619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6723b1a-7dfe-45b6-be48-1a932e03758c None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36, user.id=491e9b4bd37543c980fcb674b5e54a7b) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:41.944937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6723b1a-7dfe-45b6-be48-1a932e03758c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:41.946475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6723b1a-7dfe-45b6-be48-1a932e03758c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:41.953391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 133/266] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:41 2022] GET /identity/v3/role_assignments?user.id=491e9b4bd37543c980fcb674b5e54a7b&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 608 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:42.104242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926661-426c-413b-9079-a996a8033163 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:42.104547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926661-426c-413b-9079-a996a8033163 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:42.104812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926661-426c-413b-9079-a996a8033163 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:42.106229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 134/267] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:42.127916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa191122-2ad9-430d-bdf0-a7353ad9d23b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:42.140297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2f96a-3e82-4f1f-b231-e8a0149bd658 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:42.140596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2f96a-3e82-4f1f-b231-e8a0149bd658 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:42.140856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2f96a-3e82-4f1f-b231-e8a0149bd658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:42.165849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ad898-8ea2-4475-b6ee-8ba6e2973c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:42.167201 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ad898-8ea2-4475-b6ee-8ba6e2973c37 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:42.167576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-15e2f96a-3e82-4f1f-b231-e8a0149bd658 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:42.167886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ad898-8ea2-4475-b6ee-8ba6e2973c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:42.168327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ad898-8ea2-4475-b6ee-8ba6e2973c37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:42.168792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-15e2f96a-3e82-4f1f-b231-e8a0149bd658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:42.198639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:42.297298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ad898-8ea2-4475-b6ee-8ba6e2973c37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:42.299231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ad898-8ea2-4475-b6ee-8ba6e2973c37 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:42.303566 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 135/268] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:42.377833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2f96a-3e82-4f1f-b231-e8a0149bd658 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:42.381942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:42.482828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 134/269] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 09:44:42.492033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ad898-8ea2-4475-b6ee-8ba6e2973c37 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:42.563143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:42.611004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ad898-8ea2-4475-b6ee-8ba6e2973c37 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:42.611839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2f96a-3e82-4f1f-b231-e8a0149bd658 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:42.657994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf36be-184f-4aa4-bc2d-69c2c5918881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:42.659210 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf36be-184f-4aa4-bc2d-69c2c5918881 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:42.659819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf36be-184f-4aa4-bc2d-69c2c5918881 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:42.660236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf36be-184f-4aa4-bc2d-69c2c5918881 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:42.662515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf36be-184f-4aa4-bc2d-69c2c5918881 None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:42.665430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf36be-184f-4aa4-bc2d-69c2c5918881 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:42.668226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 135/270] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:42 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:42.675701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf36be-184f-4aa4-bc2d-69c2c5918881 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:42.690459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea807a2-7a85-4404-8f7a-6352a64e4d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:42.691563 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea807a2-7a85-4404-8f7a-6352a64e4d2c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:42.691986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea807a2-7a85-4404-8f7a-6352a64e4d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:42.692399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea807a2-7a85-4404-8f7a-6352a64e4d2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:42.696022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea807a2-7a85-4404-8f7a-6352a64e4d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:42.697694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea807a2-7a85-4404-8f7a-6352a64e4d2c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:42.701699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 136/271] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:42.705383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea807a2-7a85-4404-8f7a-6352a64e4d2c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:42.817879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf36be-184f-4aa4-bc2d-69c2c5918881 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:42.846967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:42.855389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:42.855948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f16c7-9e59-4f44-834a-a513acbd040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:42.856635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f16c7-9e59-4f44-834a-a513acbd040e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:42.856933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f16c7-9e59-4f44-834a-a513acbd040e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:42.857194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f16c7-9e59-4f44-834a-a513acbd040e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:42.858051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f16c7-9e59-4f44-834a-a513acbd040e None admin] RBAC: Adding query filter params (role.id=f819d0f29adf41a49fa045dc3c031527, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36, user.id=491e9b4bd37543c980fcb674b5e54a7b) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:42.858398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f16c7-9e59-4f44-834a-a513acbd040e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:42.860429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f16c7-9e59-4f44-834a-a513acbd040e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:42.868573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 136/272] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:42 2022] GET /identity/v3/role_assignments?user.id=491e9b4bd37543c980fcb674b5e54a7b&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=f819d0f29adf41a49fa045dc3c031527 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:42.870138 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b0930-7587-456b-9936-ee16caa18ad1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:42.870496 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b0930-7587-456b-9936-ee16caa18ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:42.870817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b0930-7587-456b-9936-ee16caa18ad1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:42.872215 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 137/273] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:42.878594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b6830-4cc8-4e3f-859e-fdeafe602cb4 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:42.878903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b6830-4cc8-4e3f-859e-fdeafe602cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:42.879185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b6830-4cc8-4e3f-859e-fdeafe602cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:42.892408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea807a2-7a85-4404-8f7a-6352a64e4d2c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:42.898321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-759b6830-4cc8-4e3f-859e-fdeafe602cb4 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:42.899614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-759b6830-4cc8-4e3f-859e-fdeafe602cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:42.953258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eef609-6c75-44d9-9830-f2c417035b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:42.953971 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eef609-6c75-44d9-9830-f2c417035b9f None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:42.954249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eef609-6c75-44d9-9830-f2c417035b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:42.954518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eef609-6c75-44d9-9830-f2c417035b9f None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:42.955393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eef609-6c75-44d9-9830-f2c417035b9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:42.959253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eef609-6c75-44d9-9830-f2c417035b9f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:43.008289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 137/274] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:44:42 2022] POST /identity/v3/users => generated 293 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:44:43.024643 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 138/275] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:43.066306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eef609-6c75-44d9-9830-f2c417035b9f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:43.197708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eef609-6c75-44d9-9830-f2c417035b9f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:43.215621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ad461-8e69-40be-ab59-8750e0899500 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:43.216157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ad461-8e69-40be-ab59-8750e0899500 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:43.216157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ad461-8e69-40be-ab59-8750e0899500 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:43.217997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 139/276] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:44:43.224507 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97d61e-b412-4aeb-9d86-265d3adadab1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:43.224794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97d61e-b412-4aeb-9d86-265d3adadab1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:43.225440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97d61e-b412-4aeb-9d86-265d3adadab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:43.232712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:43.244865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc9ecd-58c3-45d0-83dd-542c13fdcb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:43.245997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc9ecd-58c3-45d0-83dd-542c13fdcb7b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:43.246415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5f97d61e-b412-4aeb-9d86-265d3adadab1 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:43.246622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc9ecd-58c3-45d0-83dd-542c13fdcb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:43.247889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5f97d61e-b412-4aeb-9d86-265d3adadab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:43.248990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc9ecd-58c3-45d0-83dd-542c13fdcb7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:43.262931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc9ecd-58c3-45d0-83dd-542c13fdcb7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-e5bc9ecd-58c3-45d0-83dd-542c13fdcb7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:43.270834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:43.271773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:43.273071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 138/277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:43.273721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3027c-42e4-4501-a317-5e9744911ec0 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:43.273984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3027c-42e4-4501-a317-5e9744911ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:43.274721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3027c-42e4-4501-a317-5e9744911ec0 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:43.275899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 139/278] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:43.282607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d2f68-e64c-45e2-9e8e-809268940485 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:43.283020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d2f68-e64c-45e2-9e8e-809268940485 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:43.283282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d2f68-e64c-45e2-9e8e-809268940485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:43.300939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fa6d2f68-e64c-45e2-9e8e-809268940485 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:43.302246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fa6d2f68-e64c-45e2-9e8e-809268940485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:43.323090 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 140/279] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:43.323961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97d61e-b412-4aeb-9d86-265d3adadab1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:43.384344 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:43.384534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 140/280] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:43.385047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d2f68-e64c-45e2-9e8e-809268940485 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:43.442522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97d61e-b412-4aeb-9d86-265d3adadab1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:43.491590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bc5b3-37af-49df-ba17-2c5d296dd1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:43.492193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bc5b3-37af-49df-ba17-2c5d296dd1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:43.492470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bc5b3-37af-49df-ba17-2c5d296dd1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:43.492740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bc5b3-37af-49df-ba17-2c5d296dd1b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:43.493950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bc5b3-37af-49df-ba17-2c5d296dd1b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:43.494255 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bc5b3-37af-49df-ba17-2c5d296dd1b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:43.495741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bc5b3-37af-49df-ba17-2c5d296dd1b6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:43.504728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 141/281] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:43.505577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bc5b3-37af-49df-ba17-2c5d296dd1b6 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:43.513268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d2f68-e64c-45e2-9e8e-809268940485 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:43.532486 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:43.550928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34680479-4834-46a7-a543-dbff4a3b39d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:43.551835 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34680479-4834-46a7-a543-dbff4a3b39d5 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:43.552180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34680479-4834-46a7-a543-dbff4a3b39d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:43.552458 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34680479-4834-46a7-a543-dbff4a3b39d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:43.563720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34680479-4834-46a7-a543-dbff4a3b39d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-34680479-4834-46a7-a543-dbff4a3b39d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:43.571738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:43.572844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:43.573712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 141/282] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:43.574400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34680479-4834-46a7-a543-dbff4a3b39d5 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:43.648451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bc5b3-37af-49df-ba17-2c5d296dd1b6 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:43.666961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:43.700864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf864df-4790-41e6-aadc-b1fefc6da804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:43.701936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf864df-4790-41e6-aadc-b1fefc6da804 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:43.702354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf864df-4790-41e6-aadc-b1fefc6da804 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:43.703465 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf864df-4790-41e6-aadc-b1fefc6da804 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:43.721131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34680479-4834-46a7-a543-dbff4a3b39d5 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:43.725847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf864df-4790-41e6-aadc-b1fefc6da804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-2cf864df-4790-41e6-aadc-b1fefc6da804 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:43.735760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:43.736852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:43.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:43.740025 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 142/283] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:43.745456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf864df-4790-41e6-aadc-b1fefc6da804 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:43.767186 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1baa93-59ac-4903-b35f-db4edf0657b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:43.768319 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1baa93-59ac-4903-b35f-db4edf0657b0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:43.768426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1baa93-59ac-4903-b35f-db4edf0657b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:43.768698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1baa93-59ac-4903-b35f-db4edf0657b0 None admin] PATH_INFO: `/v3/users/9bd935d56d314c62b8e01d56040ed518` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:43.783975 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1baa93-59ac-4903-b35f-db4edf0657b0 None admin] RBAC: Authorizing `identity:get_user(user_id=9bd935d56d314c62b8e01d56040ed518)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:43.785477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1baa93-59ac-4903-b35f-db4edf0657b0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:43.787580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 142/284] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:43 2022] GET /identity/v3/users/9bd935d56d314c62b8e01d56040ed518 => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:43.788417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1baa93-59ac-4903-b35f-db4edf0657b0 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:43.826784 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:43.890990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf864df-4790-41e6-aadc-b1fefc6da804 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:43.903036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1baa93-59ac-4903-b35f-db4edf0657b0 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:43.931829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b3702-0894-4bdb-b583-8b6555cf7764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:43.932777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b3702-0894-4bdb-b583-8b6555cf7764 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:43.933061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b3702-0894-4bdb-b583-8b6555cf7764 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:43.933336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b3702-0894-4bdb-b583-8b6555cf7764 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:43.934250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b3702-0894-4bdb-b583-8b6555cf7764 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:43.934888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b3702-0894-4bdb-b583-8b6555cf7764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:43.936590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b3702-0894-4bdb-b583-8b6555cf7764 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:43.939450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c1b4-7462-44b3-8739-de6480c55e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:43.940312 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c1b4-7462-44b3-8739-de6480c55e75 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:43.940668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c1b4-7462-44b3-8739-de6480c55e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:43.940833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c1b4-7462-44b3-8739-de6480c55e75 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:43.941804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069c1b4-7462-44b3-8739-de6480c55e75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:43.942245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069c1b4-7462-44b3-8739-de6480c55e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:43.943785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069c1b4-7462-44b3-8739-de6480c55e75 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:43.945803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 143/285] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:43.946577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b3702-0894-4bdb-b583-8b6555cf7764 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:43.958125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 143/286] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:43.959193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c1b4-7462-44b3-8739-de6480c55e75 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:43.987489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:44.063289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b3702-0894-4bdb-b583-8b6555cf7764 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:44.075575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:44.081666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c1b4-7462-44b3-8739-de6480c55e75 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:44.098706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3da1da-0421-4163-a1e2-4a95a63e71bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:44.099379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3da1da-0421-4163-a1e2-4a95a63e71bc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:44.099660 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3da1da-0421-4163-a1e2-4a95a63e71bc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:44.100294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3da1da-0421-4163-a1e2-4a95a63e71bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:44.101131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3da1da-0421-4163-a1e2-4a95a63e71bc None admin] RBAC: Adding query filter params (role.id=f7f18826d4b44c1284ac31e25088afbd, user.id=9bd935d56d314c62b8e01d56040ed518, scope.system=all) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:44.101443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3da1da-0421-4163-a1e2-4a95a63e71bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:44.102946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3da1da-0421-4163-a1e2-4a95a63e71bc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:44.108719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 144/287] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jun 29 09:44:43 2022] GET /identity/v3/role_assignments?user.id=9bd935d56d314c62b8e01d56040ed518&scope.system=all&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:44.109593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3da1da-0421-4163-a1e2-4a95a63e71bc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:44.122147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b378c6-40b7-4bcc-905a-64e58ae9216d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:44.123512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b378c6-40b7-4bcc-905a-64e58ae9216d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:44.123937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b378c6-40b7-4bcc-905a-64e58ae9216d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:44.124352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b378c6-40b7-4bcc-905a-64e58ae9216d None admin] PATH_INFO: `/v3/users/b09e64f17ad54a2289b4ec03cd0608ff` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:44.125859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b378c6-40b7-4bcc-905a-64e58ae9216d None admin] RBAC: Authorizing `identity:get_user(user_id=b09e64f17ad54a2289b4ec03cd0608ff)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:44.127823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b378c6-40b7-4bcc-905a-64e58ae9216d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:44.129865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 144/288] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:43 2022] GET /identity/v3/users/b09e64f17ad54a2289b4ec03cd0608ff => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:44.139923 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b378c6-40b7-4bcc-905a-64e58ae9216d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:44.213250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:44.219298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3da1da-0421-4163-a1e2-4a95a63e71bc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:44.258426 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5538f-1c8c-4f58-acc7-61d9c7e17d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:44.259357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5538f-1c8c-4f58-acc7-61d9c7e17d6b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:44.259637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5538f-1c8c-4f58-acc7-61d9c7e17d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:44.259909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5538f-1c8c-4f58-acc7-61d9c7e17d6b None admin] PATH_INFO: `/v3/users/8f7cc1c0ff2847daa78400d1a1283db9` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:44.261172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5538f-1c8c-4f58-acc7-61d9c7e17d6b None admin] RBAC: Authorizing `identity:get_user(user_id=8f7cc1c0ff2847daa78400d1a1283db9)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:44.262698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5538f-1c8c-4f58-acc7-61d9c7e17d6b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:44.264716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 145/289] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:44 2022] GET /identity/v3/users/8f7cc1c0ff2847daa78400d1a1283db9 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:44.272398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5538f-1c8c-4f58-acc7-61d9c7e17d6b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:44.354588 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b378c6-40b7-4bcc-905a-64e58ae9216d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:44.411734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a540bdc-150d-4531-926d-524974b15a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:44.412632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a540bdc-150d-4531-926d-524974b15a4a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:44.412919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a540bdc-150d-4531-926d-524974b15a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:44.413198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a540bdc-150d-4531-926d-524974b15a4a None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:44.414996 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a540bdc-150d-4531-926d-524974b15a4a None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:44.416841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a540bdc-150d-4531-926d-524974b15a4a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:44.419548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 145/290] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:44 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:44.425775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a540bdc-150d-4531-926d-524974b15a4a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:44.474677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5538f-1c8c-4f58-acc7-61d9c7e17d6b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:44.508705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:44.533925 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734965bb-50e0-491f-8640-f983b33612be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:44.534699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734965bb-50e0-491f-8640-f983b33612be None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:44.534981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734965bb-50e0-491f-8640-f983b33612be None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:44.535301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734965bb-50e0-491f-8640-f983b33612be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:44.573888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:44.613039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734965bb-50e0-491f-8640-f983b33612be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:44.614451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734965bb-50e0-491f-8640-f983b33612be None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:44.688556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a540bdc-150d-4531-926d-524974b15a4a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:44.725629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 146/291] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 09:44:44.731944 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734965bb-50e0-491f-8640-f983b33612be None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:44.806965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:44.827309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256c370-1968-4290-807f-6c04133d4d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:44.828237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c370-1968-4290-807f-6c04133d4d0f None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:44.828521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c370-1968-4290-807f-6c04133d4d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:44.828807 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c370-1968-4290-807f-6c04133d4d0f None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/users/b09e64f17ad54a2289b4ec03cd0608ff/roles/61240f79e73148dbbfc847f44c528190` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:44.835963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256c370-1968-4290-807f-6c04133d4d0f None admin] RBAC: Authorizing `identity:create_grant(project_id=ea40190bfcd344248c8e0d7f7860ef36, user_id=b09e64f17ad54a2289b4ec03cd0608ff, role_id=61240f79e73148dbbfc847f44c528190)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:44.837451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256c370-1968-4290-807f-6c04133d4d0f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:44.837629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734965bb-50e0-491f-8640-f983b33612be None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:44.863429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 146/292] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:44:44 2022] PUT /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/users/b09e64f17ad54a2289b4ec03cd0608ff/roles/61240f79e73148dbbfc847f44c528190 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:44:44.909842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48319c0f-8177-41c6-8964-06e0ba949077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:44.910818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48319c0f-8177-41c6-8964-06e0ba949077 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:44.911096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48319c0f-8177-41c6-8964-06e0ba949077 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:44.911371 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48319c0f-8177-41c6-8964-06e0ba949077 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:44.915493 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48319c0f-8177-41c6-8964-06e0ba949077 None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:44.917157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48319c0f-8177-41c6-8964-06e0ba949077 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:44.919407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 147/293] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:44 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:44.925643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256c370-1968-4290-807f-6c04133d4d0f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:45.074995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:45.083127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256c370-1968-4290-807f-6c04133d4d0f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:45.136927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94973b5-9137-4db2-9ab6-04d128cb8f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:45.137519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94973b5-9137-4db2-9ab6-04d128cb8f9e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:45.137800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94973b5-9137-4db2-9ab6-04d128cb8f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:45.138076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94973b5-9137-4db2-9ab6-04d128cb8f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:45.138950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94973b5-9137-4db2-9ab6-04d128cb8f9e None admin] RBAC: Adding query filter params (role.id=f7f18826d4b44c1284ac31e25088afbd, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36, user.id=8f7cc1c0ff2847daa78400d1a1283db9) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:45.139264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94973b5-9137-4db2-9ab6-04d128cb8f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:45.140734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94973b5-9137-4db2-9ab6-04d128cb8f9e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:45.151552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 147/294] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:44 2022] GET /identity/v3/role_assignments?user.id=8f7cc1c0ff2847daa78400d1a1283db9&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:45.457045 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4268-6736-41f3-8871-6146577fca1f None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:45.457574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4268-6736-41f3-8871-6146577fca1f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:45.457574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4268-6736-41f3-8871-6146577fca1f None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:45.459337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 148/295] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:45.464421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1d682-b8f6-4d16-a9a0-7f8c7a8ff553 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:45.464864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1d682-b8f6-4d16-a9a0-7f8c7a8ff553 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:45.464974 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1d682-b8f6-4d16-a9a0-7f8c7a8ff553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:45.490918 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-e6f1d682-b8f6-4d16-a9a0-7f8c7a8ff553 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:45.492118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e6f1d682-b8f6-4d16-a9a0-7f8c7a8ff553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:45.575911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 148/296] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:45.599487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa4268-6736-41f3-8871-6146577fca1f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:45.719593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e4ccc-57ea-4bbd-bc01-e1c27501e73c None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:45.720221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e4ccc-57ea-4bbd-bc01-e1c27501e73c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:45.720827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e4ccc-57ea-4bbd-bc01-e1c27501e73c None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:45.723815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 149/297] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:45.728694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa4268-6736-41f3-8871-6146577fca1f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:45.730725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7384a4-9a33-4a23-a421-80a50b4bb702 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:45.731210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7384a4-9a33-4a23-a421-80a50b4bb702 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:45.731675 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7384a4-9a33-4a23-a421-80a50b4bb702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:45.752557 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6f7384a4-9a33-4a23-a421-80a50b4bb702 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:45.754180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6f7384a4-9a33-4a23-a421-80a50b4bb702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:45.772167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c347fc-3f6f-4578-972e-d54f69e6ec19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:45.773153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c347fc-3f6f-4578-972e-d54f69e6ec19 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:45.773437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c347fc-3f6f-4578-972e-d54f69e6ec19 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:45.773715 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c347fc-3f6f-4578-972e-d54f69e6ec19 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:45.776310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:45.788570 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c347fc-3f6f-4578-972e-d54f69e6ec19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-c2c347fc-3f6f-4578-972e-d54f69e6ec19 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:45.798056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:45.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:45.800001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 149/298] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:45.807737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c347fc-3f6f-4578-972e-d54f69e6ec19 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:45.838654 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 150/299] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:45.857641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7384a4-9a33-4a23-a421-80a50b4bb702 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:45.878166 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:45.940131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c347fc-3f6f-4578-972e-d54f69e6ec19 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:45.969167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7384a4-9a33-4a23-a421-80a50b4bb702 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:45.982378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ebf75-7e0f-4d3d-a792-332ce740e67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:45.983397 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ebf75-7e0f-4d3d-a792-332ce740e67c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:45.983872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ebf75-7e0f-4d3d-a792-332ce740e67c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:45.984347 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ebf75-7e0f-4d3d-a792-332ce740e67c None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:45.985306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ebf75-7e0f-4d3d-a792-332ce740e67c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:45.985817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ebf75-7e0f-4d3d-a792-332ce740e67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:45.987497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ebf75-7e0f-4d3d-a792-332ce740e67c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:45.996812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 150/300] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:46.003695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ebf75-7e0f-4d3d-a792-332ce740e67c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:46.024707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf0e33-36cf-4e92-a85b-3b546f57ad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:46.026169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0e33-36cf-4e92-a85b-3b546f57ad9d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:46.026632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0e33-36cf-4e92-a85b-3b546f57ad9d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:46.027095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0e33-36cf-4e92-a85b-3b546f57ad9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:46.028609 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf0e33-36cf-4e92-a85b-3b546f57ad9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:46.030265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf0e33-36cf-4e92-a85b-3b546f57ad9d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:46.038036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 151/301] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:46.044590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf0e33-36cf-4e92-a85b-3b546f57ad9d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:46.082060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:46.143423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ebf75-7e0f-4d3d-a792-332ce740e67c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:46.163733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:46.181731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c37e7-4c77-4593-a89a-571ad9d2d1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:46.183037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c37e7-4c77-4593-a89a-571ad9d2d1ca None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:46.183527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c37e7-4c77-4593-a89a-571ad9d2d1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:46.184005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c37e7-4c77-4593-a89a-571ad9d2d1ca None admin] PATH_INFO: `/v3/users/8c283fb281d84634a5a03349ce100729` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:46.184512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf0e33-36cf-4e92-a85b-3b546f57ad9d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:46.199794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c37e7-4c77-4593-a89a-571ad9d2d1ca None admin] RBAC: Authorizing `identity:get_user(user_id=8c283fb281d84634a5a03349ce100729)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:46.201609 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c37e7-4c77-4593-a89a-571ad9d2d1ca None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:46.204177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 151/302] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:46 2022] GET /identity/v3/users/8c283fb281d84634a5a03349ce100729 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:46.211639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c37e7-4c77-4593-a89a-571ad9d2d1ca None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:46.227696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf80b80-7482-43c3-a6c4-f52d8676c682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:46.228774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf80b80-7482-43c3-a6c4-f52d8676c682 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:46.229304 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf80b80-7482-43c3-a6c4-f52d8676c682 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:46.229800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf80b80-7482-43c3-a6c4-f52d8676c682 None admin] PATH_INFO: `/v3/groups` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:46.230729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf80b80-7482-43c3-a6c4-f52d8676c682 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:46.232479 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf80b80-7482-43c3-a6c4-f52d8676c682 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:46.257807 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 152/303] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Jun 29 09:44:46 2022] POST /identity/v3/groups => generated 229 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:44:46.339272 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c37e7-4c77-4593-a89a-571ad9d2d1ca None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:46.357655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:46.364308 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042d940-3613-45ee-859f-6a2a1d2dedee None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:46.364592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042d940-3613-45ee-859f-6a2a1d2dedee None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:46.364870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042d940-3613-45ee-859f-6a2a1d2dedee None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:46.366610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 153/304] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:44:46.372396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a633c-a284-47e1-a804-a78e07ae2e94 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:46.372644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a633c-a284-47e1-a804-a78e07ae2e94 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:46.372886 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a633c-a284-47e1-a804-a78e07ae2e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:46.385515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edae43-8f8a-4498-ab7c-4238d8308a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:46.387128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edae43-8f8a-4498-ab7c-4238d8308a18 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:46.387771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edae43-8f8a-4498-ab7c-4238d8308a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:46.388395 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edae43-8f8a-4498-ab7c-4238d8308a18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:46.390027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59edae43-8f8a-4498-ab7c-4238d8308a18 None admin] RBAC: Adding query filter params (role.id=61240f79e73148dbbfc847f44c528190, user.id=8c283fb281d84634a5a03349ce100729, scope.system=all) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:46.390793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59edae43-8f8a-4498-ab7c-4238d8308a18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:46.392186 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-672a633c-a284-47e1-a804-a78e07ae2e94 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:46.393618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-672a633c-a284-47e1-a804-a78e07ae2e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:46.394038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59edae43-8f8a-4498-ab7c-4238d8308a18 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:46.404006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 152/305] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jun 29 09:44:46 2022] GET /identity/v3/role_assignments?user.id=8c283fb281d84634a5a03349ce100729&scope.system=all&role.id=61240f79e73148dbbfc847f44c528190 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:46.482453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 154/306] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:46.505398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edae43-8f8a-4498-ab7c-4238d8308a18 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:46.613678 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edae43-8f8a-4498-ab7c-4238d8308a18 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:46.648950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637183a0-b114-4ef3-9003-9663bcab606e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:46.649761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637183a0-b114-4ef3-9003-9663bcab606e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:46.650040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637183a0-b114-4ef3-9003-9663bcab606e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:46.650307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637183a0-b114-4ef3-9003-9663bcab606e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:46.661229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637183a0-b114-4ef3-9003-9663bcab606e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-637183a0-b114-4ef3-9003-9663bcab606e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:46.668979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:46.669954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:46.670759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 153/307] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:46.676670 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a633c-a284-47e1-a804-a78e07ae2e94 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:46.757505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:46.781741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a633c-a284-47e1-a804-a78e07ae2e94 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:46.819897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c2bb3-3f10-4035-b01c-ec8ec43f82af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:46.820597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c2bb3-3f10-4035-b01c-ec8ec43f82af None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:46.820876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c2bb3-3f10-4035-b01c-ec8ec43f82af None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:46.821149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c2bb3-3f10-4035-b01c-ec8ec43f82af None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:46.821908 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c2bb3-3f10-4035-b01c-ec8ec43f82af None admin] RBAC: Adding query filter params (name=member) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:46.822215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c2bb3-3f10-4035-b01c-ec8ec43f82af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:46.823647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c2bb3-3f10-4035-b01c-ec8ec43f82af None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:46.832892 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 155/308] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:46.838587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637183a0-b114-4ef3-9003-9663bcab606e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:46.881857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:46.949018 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637183a0-b114-4ef3-9003-9663bcab606e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:46.984661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8890d32-d6b6-47c5-8db6-c3939f5ed5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:46.985917 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8890d32-d6b6-47c5-8db6-c3939f5ed5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:46.986447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8890d32-d6b6-47c5-8db6-c3939f5ed5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:46.986938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8890d32-d6b6-47c5-8db6-c3939f5ed5c2 None admin] PATH_INFO: `/v3/users/9bd935d56d314c62b8e01d56040ed518` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:46.988355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8890d32-d6b6-47c5-8db6-c3939f5ed5c2 None admin] RBAC: Authorizing `identity:get_user(user_id=9bd935d56d314c62b8e01d56040ed518)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:46.990006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8890d32-d6b6-47c5-8db6-c3939f5ed5c2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:46.992198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 154/309] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:46 2022] GET /identity/v3/users/9bd935d56d314c62b8e01d56040ed518 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:46.998063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c2bb3-3f10-4035-b01c-ec8ec43f82af None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:47.109309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c2bb3-3f10-4035-b01c-ec8ec43f82af None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:47.143992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39354c1a-146b-4da7-a5f5-100f2e1e5c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:47.144878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39354c1a-146b-4da7-a5f5-100f2e1e5c8f None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:47.145155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39354c1a-146b-4da7-a5f5-100f2e1e5c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:47.145427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39354c1a-146b-4da7-a5f5-100f2e1e5c8f None admin] PATH_INFO: `/v3/system/users/9bd935d56d314c62b8e01d56040ed518/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:47.146978 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39354c1a-146b-4da7-a5f5-100f2e1e5c8f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9bd935d56d314c62b8e01d56040ed518, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:47.148385 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39354c1a-146b-4da7-a5f5-100f2e1e5c8f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:47.162744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 156/310] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Jun 29 09:44:46 2022] PUT /identity/v3/system/users/9bd935d56d314c62b8e01d56040ed518/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:44:47.254938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce0959-ed7a-45fe-ae3e-4eb383c1b35d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:47.255567 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce0959-ed7a-45fe-ae3e-4eb383c1b35d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:47.256061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce0959-ed7a-45fe-ae3e-4eb383c1b35d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:47.258167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 155/311] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:47.263643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155e2e6-f8fc-44ce-9993-da5ce003538a None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:47.263939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155e2e6-f8fc-44ce-9993-da5ce003538a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:47.264208 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155e2e6-f8fc-44ce-9993-da5ce003538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:47.269703 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:47.286768 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-d155e2e6-f8fc-44ce-9993-da5ce003538a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:47.288348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d155e2e6-f8fc-44ce-9993-da5ce003538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:47.365605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 157/312] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:47.388494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce0959-ed7a-45fe-ae3e-4eb383c1b35d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:47.498705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce0959-ed7a-45fe-ae3e-4eb383c1b35d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:47.507388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:47.544102 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841e43e-c8b0-492a-91a0-cf03a3dba24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:47.544940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841e43e-c8b0-492a-91a0-cf03a3dba24e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:47.545215 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841e43e-c8b0-492a-91a0-cf03a3dba24e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:47.545483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841e43e-c8b0-492a-91a0-cf03a3dba24e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:47.557809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841e43e-c8b0-492a-91a0-cf03a3dba24e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-3841e43e-c8b0-492a-91a0-cf03a3dba24e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:47.565962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:47.566968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:47.567756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 156/313] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:47.573128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155e2e6-f8fc-44ce-9993-da5ce003538a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:47.684279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155e2e6-f8fc-44ce-9993-da5ce003538a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:47.717741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:47.722152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176225e6-9a31-4d7e-a0b8-3970218165b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:47.723122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176225e6-9a31-4d7e-a0b8-3970218165b5 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:47.723573 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176225e6-9a31-4d7e-a0b8-3970218165b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:47.724005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176225e6-9a31-4d7e-a0b8-3970218165b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:47.725279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176225e6-9a31-4d7e-a0b8-3970218165b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:47.726656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176225e6-9a31-4d7e-a0b8-3970218165b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:47.728280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176225e6-9a31-4d7e-a0b8-3970218165b5 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:47.738183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 158/314] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:47.744048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841e43e-c8b0-492a-91a0-cf03a3dba24e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:47.846499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841e43e-c8b0-492a-91a0-cf03a3dba24e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:47.881104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13d16f-50d3-4554-b9a4-79e5c788c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:47.881993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13d16f-50d3-4554-b9a4-79e5c788c76f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:47.882271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13d16f-50d3-4554-b9a4-79e5c788c76f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:47.882541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13d16f-50d3-4554-b9a4-79e5c788c76f None admin] PATH_INFO: `/v3/users/b09e64f17ad54a2289b4ec03cd0608ff` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:47.883791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13d16f-50d3-4554-b9a4-79e5c788c76f None admin] RBAC: Authorizing `identity:get_user(user_id=b09e64f17ad54a2289b4ec03cd0608ff)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:47.885251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13d16f-50d3-4554-b9a4-79e5c788c76f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:47.886982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 157/315] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:47 2022] GET /identity/v3/users/b09e64f17ad54a2289b4ec03cd0608ff => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:47.892606 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176225e6-9a31-4d7e-a0b8-3970218165b5 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:48.001160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176225e6-9a31-4d7e-a0b8-3970218165b5 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:48.037869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7aafa2-bf8b-456c-a9b2-ad30671482a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:48.038926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7aafa2-bf8b-456c-a9b2-ad30671482a0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:48.039504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7aafa2-bf8b-456c-a9b2-ad30671482a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:48.039977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7aafa2-bf8b-456c-a9b2-ad30671482a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:48.107077 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:48.116293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7aafa2-bf8b-456c-a9b2-ad30671482a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:48.118101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7aafa2-bf8b-456c-a9b2-ad30671482a0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:48.231741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 159/316] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:48.237646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13d16f-50d3-4554-b9a4-79e5c788c76f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:48.273378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:48.351045 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13d16f-50d3-4554-b9a4-79e5c788c76f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:48.387321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4aa45-0a69-46ca-b984-a48adf108d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:48.388205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4aa45-0a69-46ca-b984-a48adf108d30 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:48.388487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4aa45-0a69-46ca-b984-a48adf108d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:48.388760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4aa45-0a69-46ca-b984-a48adf108d30 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:48.389965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4aa45-0a69-46ca-b984-a48adf108d30 None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:48.391454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4aa45-0a69-46ca-b984-a48adf108d30 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:48.393410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 158/317] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:48 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:48.399155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7aafa2-bf8b-456c-a9b2-ad30671482a0 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:48.444185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e8589-f301-4d81-bdda-5403a96d89b1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:48.444813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e8589-f301-4d81-bdda-5403a96d89b1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:48.445307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e8589-f301-4d81-bdda-5403a96d89b1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:48.447223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 159/318] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:48.453214 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0e087-b1a3-4f24-93a2-d861df740c21 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:48.454058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0e087-b1a3-4f24-93a2-d861df740c21 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:48.454848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0e087-b1a3-4f24-93a2-d861df740c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:48.475686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1bc0e087-b1a3-4f24-93a2-d861df740c21 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:48.477232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1bc0e087-b1a3-4f24-93a2-d861df740c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:48.515069 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:48.523270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7aafa2-bf8b-456c-a9b2-ad30671482a0 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:48.561720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a94416-a81c-4808-b2ee-bd3f77a9104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:48.561905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 160/319] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:48.570586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a94416-a81c-4808-b2ee-bd3f77a9104c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:48.570586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a94416-a81c-4808-b2ee-bd3f77a9104c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:48.570586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a94416-a81c-4808-b2ee-bd3f77a9104c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:48.570586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a94416-a81c-4808-b2ee-bd3f77a9104c None admin] RBAC: Adding query filter params (role.id=61240f79e73148dbbfc847f44c528190, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36, user.id=b09e64f17ad54a2289b4ec03cd0608ff) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:48.570586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a94416-a81c-4808-b2ee-bd3f77a9104c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:48.570586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a94416-a81c-4808-b2ee-bd3f77a9104c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:48.576767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 160/320] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:44:48 2022] GET /identity/v3/role_assignments?user.id=b09e64f17ad54a2289b4ec03cd0608ff&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=61240f79e73148dbbfc847f44c528190 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:48.588498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0e087-b1a3-4f24-93a2-d861df740c21 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:48.621124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdf690-4a40-4ea9-bc77-4dfcbfe3ddbe None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:48.621755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdf690-4a40-4ea9-bc77-4dfcbfe3ddbe None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:48.622212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdf690-4a40-4ea9-bc77-4dfcbfe3ddbe None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:48.623929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 161/321] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:44:48.628671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:48.630590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea9e99-cedf-4fb8-bf84-25746ed88ffd None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:48.631085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea9e99-cedf-4fb8-bf84-25746ed88ffd None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:48.631555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea9e99-cedf-4fb8-bf84-25746ed88ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:48.649365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-07ea9e99-cedf-4fb8-bf84-25746ed88ffd None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:48.651270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-07ea9e99-cedf-4fb8-bf84-25746ed88ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:48.707263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0e087-b1a3-4f24-93a2-d861df740c21 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:48.739085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 162/322] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:48.750200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc95e-2f42-425d-8900-3e35e17b56d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:48.751061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc95e-2f42-425d-8900-3e35e17b56d3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:48.751340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc95e-2f42-425d-8900-3e35e17b56d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:48.751608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc95e-2f42-425d-8900-3e35e17b56d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:48.752919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fc95e-2f42-425d-8900-3e35e17b56d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:48.754415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fc95e-2f42-425d-8900-3e35e17b56d3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:48.756491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 161/323] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:44:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:48.762147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc95e-2f42-425d-8900-3e35e17b56d3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:48.762457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea9e99-cedf-4fb8-bf84-25746ed88ffd None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:48.878728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:48.899055 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:48.899375 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea9e99-cedf-4fb8-bf84-25746ed88ffd None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:48.902310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc95e-2f42-425d-8900-3e35e17b56d3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:48.934441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe57ba2-a0e6-4be2-9d16-c437f4e100ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:48.935300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe57ba2-a0e6-4be2-9d16-c437f4e100ff None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:48.935583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe57ba2-a0e6-4be2-9d16-c437f4e100ff None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:48.935855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe57ba2-a0e6-4be2-9d16-c437f4e100ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:48.941544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd72623-2fdd-42be-abc1-37396d72aa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:48.942381 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd72623-2fdd-42be-abc1-37396d72aa91 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:48.942872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd72623-2fdd-42be-abc1-37396d72aa91 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:48.943152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd72623-2fdd-42be-abc1-37396d72aa91 None admin] PATH_INFO: `/v3/groups` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:48.943967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd72623-2fdd-42be-abc1-37396d72aa91 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:48.945415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd72623-2fdd-42be-abc1-37396d72aa91 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:48.946393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe57ba2-a0e6-4be2-9d16-c437f4e100ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-bbe57ba2-a0e6-4be2-9d16-c437f4e100ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:48.954939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:48.956068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:48.957225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 163/324] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:48.962806 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe57ba2-a0e6-4be2-9d16-c437f4e100ff None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:48.978716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 162/325] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Jun 29 09:44:48 2022] POST /identity/v3/groups => generated 226 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:44:49.093688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe57ba2-a0e6-4be2-9d16-c437f4e100ff None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:49.133836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba7a58-44b6-41ee-8eee-5bfdb8139897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:49.134455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba7a58-44b6-41ee-8eee-5bfdb8139897 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:49.134767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba7a58-44b6-41ee-8eee-5bfdb8139897 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:49.135044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba7a58-44b6-41ee-8eee-5bfdb8139897 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:49.135808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba7a58-44b6-41ee-8eee-5bfdb8139897 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:49.136119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba7a58-44b6-41ee-8eee-5bfdb8139897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:49.137507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba7a58-44b6-41ee-8eee-5bfdb8139897 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:49.146853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 164/326] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:49.152720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd72623-2fdd-42be-abc1-37396d72aa91 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:49.262090 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd72623-2fdd-42be-abc1-37396d72aa91 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:49.303859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b79ec7-e75a-4ecc-b6bb-c0b169a55795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:49.304861 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b79ec7-e75a-4ecc-b6bb-c0b169a55795 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:49.305143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b79ec7-e75a-4ecc-b6bb-c0b169a55795 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:49.305419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b79ec7-e75a-4ecc-b6bb-c0b169a55795 None admin] PATH_INFO: `/v3/users/8c283fb281d84634a5a03349ce100729` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:49.305974 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8d00f-9aca-41c0-beff-7c7b398af2c0 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:49.306268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8d00f-9aca-41c0-beff-7c7b398af2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:49.306550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8d00f-9aca-41c0-beff-7c7b398af2c0 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:49.306718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b79ec7-e75a-4ecc-b6bb-c0b169a55795 None admin] RBAC: Authorizing `identity:get_user(user_id=8c283fb281d84634a5a03349ce100729)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:49.308148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 165/327] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:49.308258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b79ec7-e75a-4ecc-b6bb-c0b169a55795 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:49.309989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 163/328] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:49 2022] GET /identity/v3/users/8c283fb281d84634a5a03349ce100729 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:49.314767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac7c45-9802-482e-be0b-fe07f7eab4f8 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:49.315176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac7c45-9802-482e-be0b-fe07f7eab4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:49.315351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b79ec7-e75a-4ecc-b6bb-c0b169a55795 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:49.315458 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac7c45-9802-482e-be0b-fe07f7eab4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:49.341055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-e2ac7c45-9802-482e-be0b-fe07f7eab4f8 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:49.342601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e2ac7c45-9802-482e-be0b-fe07f7eab4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:49.347417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:49.347878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:49.433003 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 166/329] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:49.448168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b79ec7-e75a-4ecc-b6bb-c0b169a55795 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:49.458869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac7c45-9802-482e-be0b-fe07f7eab4f8 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:49.496756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5f2fc-b3be-48f7-98df-0ec9ab0bd269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:49.498049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5f2fc-b3be-48f7-98df-0ec9ab0bd269 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:49.498528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5f2fc-b3be-48f7-98df-0ec9ab0bd269 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:49.499026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5f2fc-b3be-48f7-98df-0ec9ab0bd269 None admin] PATH_INFO: `/v3/system/users/8c283fb281d84634a5a03349ce100729/roles/61240f79e73148dbbfc847f44c528190` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:49.506641 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5f2fc-b3be-48f7-98df-0ec9ab0bd269 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8c283fb281d84634a5a03349ce100729, role_id=61240f79e73148dbbfc847f44c528190)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:49.508584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5f2fc-b3be-48f7-98df-0ec9ab0bd269 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:49.525821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 164/330] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Jun 29 09:44:49 2022] PUT /identity/v3/system/users/8c283fb281d84634a5a03349ce100729/roles/61240f79e73148dbbfc847f44c528190 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:44:49.590756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac7c45-9802-482e-be0b-fe07f7eab4f8 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:49.633272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e757271-3506-4920-bc82-66dc04353dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:49.634180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757271-3506-4920-bc82-66dc04353dba None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:49.634463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757271-3506-4920-bc82-66dc04353dba None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:49.634767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757271-3506-4920-bc82-66dc04353dba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:49.644701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:49.648758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e757271-3506-4920-bc82-66dc04353dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-7e757271-3506-4920-bc82-66dc04353dba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:49.657564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:49.658658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:49.659644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 167/331] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:49.666081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5f2fc-b3be-48f7-98df-0ec9ab0bd269 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:49.769897 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:49.779470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5f2fc-b3be-48f7-98df-0ec9ab0bd269 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:49.816165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192efa4-e0cd-49c4-bed3-2646c0a46983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:49.816855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192efa4-e0cd-49c4-bed3-2646c0a46983 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:49.817139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192efa4-e0cd-49c4-bed3-2646c0a46983 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:49.817413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192efa4-e0cd-49c4-bed3-2646c0a46983 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:49.818184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192efa4-e0cd-49c4-bed3-2646c0a46983 None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:49.818491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192efa4-e0cd-49c4-bed3-2646c0a46983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:49.819946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192efa4-e0cd-49c4-bed3-2646c0a46983 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:49.829251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 165/332] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:49.835090 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e757271-3506-4920-bc82-66dc04353dba None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:49.959573 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e757271-3506-4920-bc82-66dc04353dba None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:50.004269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccc003-6cc2-44ea-9ba4-7855e5553cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:50.005166 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccc003-6cc2-44ea-9ba4-7855e5553cdf None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:50.005446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccc003-6cc2-44ea-9ba4-7855e5553cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:50.005717 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccc003-6cc2-44ea-9ba4-7855e5553cdf None admin] PATH_INFO: `/v3/users/9bd935d56d314c62b8e01d56040ed518` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:50.010030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ccc003-6cc2-44ea-9ba4-7855e5553cdf None admin] RBAC: Authorizing `identity:get_user(user_id=9bd935d56d314c62b8e01d56040ed518)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:50.011607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ccc003-6cc2-44ea-9ba4-7855e5553cdf None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:50.013540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 168/333] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:49 2022] GET /identity/v3/users/9bd935d56d314c62b8e01d56040ed518 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:50.019461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192efa4-e0cd-49c4-bed3-2646c0a46983 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:50.214962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192efa4-e0cd-49c4-bed3-2646c0a46983 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:50.275392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24897106-ca43-40bf-ad2a-66c7a87dbe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:50.276086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897106-ca43-40bf-ad2a-66c7a87dbe5a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:50.276369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897106-ca43-40bf-ad2a-66c7a87dbe5a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:50.276642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897106-ca43-40bf-ad2a-66c7a87dbe5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:50.277512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24897106-ca43-40bf-ad2a-66c7a87dbe5a None admin] RBAC: Adding query filter params (role.id=f7f18826d4b44c1284ac31e25088afbd, user.id=9bd935d56d314c62b8e01d56040ed518, scope.system=all) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:50.277833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24897106-ca43-40bf-ad2a-66c7a87dbe5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:50.279353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24897106-ca43-40bf-ad2a-66c7a87dbe5a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:50.295599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 166/334] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jun 29 09:44:50 2022] GET /identity/v3/role_assignments?user.id=9bd935d56d314c62b8e01d56040ed518&scope.system=all&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 510 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:51.872428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdec30-93f7-40c8-9215-d4b72535a25d None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:51.873462 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdec30-93f7-40c8-9215-d4b72535a25d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:51.873977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdec30-93f7-40c8-9215-d4b72535a25d None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:51.875958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 169/335] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:51.883168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef29397-55cb-4cdb-b72c-64d27eb98648 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:51.884074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef29397-55cb-4cdb-b72c-64d27eb98648 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:51.884544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef29397-55cb-4cdb-b72c-64d27eb98648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:51.907623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-cef29397-55cb-4cdb-b72c-64d27eb98648 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:51.909718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd1ce7-a37c-4ff8-a28f-8c521dfae057 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:51.910221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd1ce7-a37c-4ff8-a28f-8c521dfae057 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:51.910739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd1ce7-a37c-4ff8-a28f-8c521dfae057 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:51.912334 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 170/336] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:51.913772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-cef29397-55cb-4cdb-b72c-64d27eb98648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:51.917963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441aaa1-913e-4bbb-bd6e-0979517ec235 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:51.918093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441aaa1-913e-4bbb-bd6e-0979517ec235 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:51.918266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441aaa1-913e-4bbb-bd6e-0979517ec235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:51.923423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:51.947396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-3441aaa1-913e-4bbb-bd6e-0979517ec235 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:51.949038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3441aaa1-913e-4bbb-bd6e-0979517ec235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:52.020937 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 167/337] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:52.054283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef29397-55cb-4cdb-b72c-64d27eb98648 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:52.073178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:52.081170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 171/338] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:52.116356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441aaa1-913e-4bbb-bd6e-0979517ec235 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:52.239640 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef29397-55cb-4cdb-b72c-64d27eb98648 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:52.318740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1a954-d698-4c03-a8db-b18b8199ea34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:52.319896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1a954-d698-4c03-a8db-b18b8199ea34 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:52.320298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1a954-d698-4c03-a8db-b18b8199ea34 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:52.320688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1a954-d698-4c03-a8db-b18b8199ea34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:52.339898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1a954-d698-4c03-a8db-b18b8199ea34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-25a1a954-d698-4c03-a8db-b18b8199ea34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:52.349543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:52.350873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:52.351664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:52.352269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441aaa1-913e-4bbb-bd6e-0979517ec235 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:52.352349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 168/339] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:44:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:52.358605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1a954-d698-4c03-a8db-b18b8199ea34 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:52.362896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:52.420090 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51478f52-20d6-4620-b5d7-ef144e61eb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:52.425743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51478f52-20d6-4620-b5d7-ef144e61eb15 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:52.425743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51478f52-20d6-4620-b5d7-ef144e61eb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:52.425743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51478f52-20d6-4620-b5d7-ef144e61eb15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:52.448049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51478f52-20d6-4620-b5d7-ef144e61eb15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-51478f52-20d6-4620-b5d7-ef144e61eb15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:52.456092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:52.457019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:52.457788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 172/340] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:52.466283 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51478f52-20d6-4620-b5d7-ef144e61eb15 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:52.564756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1a954-d698-4c03-a8db-b18b8199ea34 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:52.594953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:52.617517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b5452-2397-4b29-b38c-75d887de0437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:52.618214 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5452-2397-4b29-b38c-75d887de0437 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:52.618501 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5452-2397-4b29-b38c-75d887de0437 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:52.619005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5452-2397-4b29-b38c-75d887de0437 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:52.619784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b5452-2397-4b29-b38c-75d887de0437 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:52.620097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b5452-2397-4b29-b38c-75d887de0437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:52.621516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b5452-2397-4b29-b38c-75d887de0437 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:52.636761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 169/341] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:44:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:52.643274 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b5452-2397-4b29-b38c-75d887de0437 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:52.715218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51478f52-20d6-4620-b5d7-ef144e61eb15 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:52.735066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:52.780781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b5452-2397-4b29-b38c-75d887de0437 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:52.801360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf1116-d457-4a42-94e7-e464151bf449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:52.802057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf1116-d457-4a42-94e7-e464151bf449 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:52.802339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf1116-d457-4a42-94e7-e464151bf449 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:52.802650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf1116-d457-4a42-94e7-e464151bf449 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:52.803414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf1116-d457-4a42-94e7-e464151bf449 None admin] RBAC: Adding query filter params (name=member) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:52.803722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf1116-d457-4a42-94e7-e464151bf449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:52.805222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf1116-d457-4a42-94e7-e464151bf449 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:52.826123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 173/342] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:44:52.829151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56f007-0d8f-4b37-b035-9ee17229b792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:52.829824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56f007-0d8f-4b37-b035-9ee17229b792 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:52.830166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56f007-0d8f-4b37-b035-9ee17229b792 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:52.830447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56f007-0d8f-4b37-b035-9ee17229b792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:52.839005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf1116-d457-4a42-94e7-e464151bf449 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:52.954070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56f007-0d8f-4b37-b035-9ee17229b792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:52.955774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56f007-0d8f-4b37-b035-9ee17229b792 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:53.036347 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf1116-d457-4a42-94e7-e464151bf449 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:53.069102 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:53.073401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:53.092625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478b4da-2021-42a1-a707-cab3744ea701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:53.093311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478b4da-2021-42a1-a707-cab3744ea701 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.093594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478b4da-2021-42a1-a707-cab3744ea701 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.093872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478b4da-2021-42a1-a707-cab3744ea701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.118518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 170/343] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 09:44:53.120030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719a615-c1f2-4321-b61f-21cd1349719d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.120480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719a615-c1f2-4321-b61f-21cd1349719d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.120933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719a615-c1f2-4321-b61f-21cd1349719d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.122401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 171/344] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:53.123781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5e6a5-b335-489a-8645-85040c105076 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.124277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5e6a5-b335-489a-8645-85040c105076 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.124754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5e6a5-b335-489a-8645-85040c105076 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.126527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 172/345] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:53.128324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726bdb1-01ca-4503-acfa-c0f078cf430b None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.128568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726bdb1-01ca-4503-acfa-c0f078cf430b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.128845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726bdb1-01ca-4503-acfa-c0f078cf430b None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.130291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 173/346] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:53.130952 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726bdb1-01ca-4503-acfa-c0f078cf430b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:53.200149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478b4da-2021-42a1-a707-cab3744ea701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:53.203040 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478b4da-2021-42a1-a707-cab3744ea701 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:53.263229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726bdb1-01ca-4503-acfa-c0f078cf430b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:53.301514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385bb81-4669-4fad-a414-8472ab6c0c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:53.302403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385bb81-4669-4fad-a414-8472ab6c0c97 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.302891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385bb81-4669-4fad-a414-8472ab6c0c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.302994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385bb81-4669-4fad-a414-8472ab6c0c97 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.304191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385bb81-4669-4fad-a414-8472ab6c0c97 None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:53.305670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385bb81-4669-4fad-a414-8472ab6c0c97 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:53.311990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 174/347] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:53 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:53.313356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf50232-96e7-45df-83a6-327d7aa52782 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.313643 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf50232-96e7-45df-83a6-327d7aa52782 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.313926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf50232-96e7-45df-83a6-327d7aa52782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.321426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 174/348] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 09:44:53.323069 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c439d1-b853-4b88-933e-8d5ebad5de49 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.323576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c439d1-b853-4b88-933e-8d5ebad5de49 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.324024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c439d1-b853-4b88-933e-8d5ebad5de49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.332082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ecf50232-96e7-45df-83a6-327d7aa52782 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:53.333289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ecf50232-96e7-45df-83a6-327d7aa52782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:53.344468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-35c439d1-b853-4b88-933e-8d5ebad5de49 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:53.345062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-35c439d1-b853-4b88-933e-8d5ebad5de49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:53.360991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:53.365688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:53.420984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 175/349] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:53.424292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a58ec-e425-4107-9b91-936289ae45f6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.424655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a58ec-e425-4107-9b91-936289ae45f6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.425000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a58ec-e425-4107-9b91-936289ae45f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.425637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 175/350] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:53.426854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c439d1-b853-4b88-933e-8d5ebad5de49 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:53.449228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-586a58ec-e425-4107-9b91-936289ae45f6 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:53.450442 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-586a58ec-e425-4107-9b91-936289ae45f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:53.553220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 176/351] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:53.554088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a58ec-e425-4107-9b91-936289ae45f6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:53.579931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c439d1-b853-4b88-933e-8d5ebad5de49 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:53.623350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5576f7-108e-4057-9ea3-01d65e4349c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:53.624144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5576f7-108e-4057-9ea3-01d65e4349c2 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.624426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5576f7-108e-4057-9ea3-01d65e4349c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.625189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5576f7-108e-4057-9ea3-01d65e4349c2 None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.636113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5576f7-108e-4057-9ea3-01d65e4349c2 None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:53.637580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5576f7-108e-4057-9ea3-01d65e4349c2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:53.639582 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 176/352] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:44:53 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:53.640516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5576f7-108e-4057-9ea3-01d65e4349c2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:53.661278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:53.683373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:53.695431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a58ec-e425-4107-9b91-936289ae45f6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:53.738158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5cd2f-6f60-4c90-87e4-12440f963ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:53.739400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5cd2f-6f60-4c90-87e4-12440f963ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.739882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5cd2f-6f60-4c90-87e4-12440f963ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.740335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5cd2f-6f60-4c90-87e4-12440f963ee3 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.741753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5cd2f-6f60-4c90-87e4-12440f963ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:53.744597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5cd2f-6f60-4c90-87e4-12440f963ee3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:53.746931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 177/353] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:53 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:53.747913 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5cd2f-6f60-4c90-87e4-12440f963ee3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:53.774019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5576f7-108e-4057-9ea3-01d65e4349c2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:53.814391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce551c5-317e-404b-9825-a3a055e132ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:53.815508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce551c5-317e-404b-9825-a3a055e132ef None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.815943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce551c5-317e-404b-9825-a3a055e132ef None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.816372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce551c5-317e-404b-9825-a3a055e132ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.828871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce551c5-317e-404b-9825-a3a055e132ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-dce551c5-317e-404b-9825-a3a055e132ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:53.837447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:53.839682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 09:44:53.840748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:53.843278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 177/354] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:53.844226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce551c5-317e-404b-9825-a3a055e132ef None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:53.876561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5cd2f-6f60-4c90-87e4-12440f963ee3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:53.926541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f4037-9812-4f59-8daa-77cae3dd0e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:53.927400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f4037-9812-4f59-8daa-77cae3dd0e7a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:53.927658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f4037-9812-4f59-8daa-77cae3dd0e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:53.927927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f4037-9812-4f59-8daa-77cae3dd0e7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:53.941019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f4037-9812-4f59-8daa-77cae3dd0e7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:53.947262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-277f4037-9812-4f59-8daa-77cae3dd0e7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:53.950003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:53.950929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 178/355] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:53.952357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f4037-9812-4f59-8daa-77cae3dd0e7a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:53.979022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce551c5-317e-404b-9825-a3a055e132ef None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:53.981961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:54.021609 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29129c-a045-4886-a330-2aa302c688dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:54.022447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29129c-a045-4886-a330-2aa302c688dc None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:54.022758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29129c-a045-4886-a330-2aa302c688dc None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:54.023040 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29129c-a045-4886-a330-2aa302c688dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:54.037351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29129c-a045-4886-a330-2aa302c688dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-6c29129c-a045-4886-a330-2aa302c688dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:54.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:54.046440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:54.047286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 178/356] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:44:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:54.047972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29129c-a045-4886-a330-2aa302c688dc None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:54.086595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f4037-9812-4f59-8daa-77cae3dd0e7a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:54.130117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb718b-3fd2-4fa4-b907-6e262cf52a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:54.130751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb718b-3fd2-4fa4-b907-6e262cf52a7a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:54.131033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb718b-3fd2-4fa4-b907-6e262cf52a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:54.131308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb718b-3fd2-4fa4-b907-6e262cf52a7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:54.132155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb718b-3fd2-4fa4-b907-6e262cf52a7a None admin] RBAC: Adding query filter params (group.id=8327f367eb2848039fa0c9e8eae691d0, role.id=f819d0f29adf41a49fa045dc3c031527, scope.project.id=c65d427580b743dba5cb657edb44cbbc) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:54.132466 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb718b-3fd2-4fa4-b907-6e262cf52a7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:54.134520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb718b-3fd2-4fa4-b907-6e262cf52a7a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:54.141073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 179/357] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 09:44:53 2022] GET /identity/v3/role_assignments?group.id=8327f367eb2848039fa0c9e8eae691d0&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=f819d0f29adf41a49fa045dc3c031527 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:54.141740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb718b-3fd2-4fa4-b907-6e262cf52a7a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:54.184616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29129c-a045-4886-a330-2aa302c688dc None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:54.235150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bbcc0-01d0-4d09-b4ce-4f82d15c6e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:54.236531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bbcc0-01d0-4d09-b4ce-4f82d15c6e07 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:54.237179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bbcc0-01d0-4d09-b4ce-4f82d15c6e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:54.237967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bbcc0-01d0-4d09-b4ce-4f82d15c6e07 None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:54.240727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bbcc0-01d0-4d09-b4ce-4f82d15c6e07 None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:54.242226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bbcc0-01d0-4d09-b4ce-4f82d15c6e07 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:54.244155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 179/358] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:44:54 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:54.245459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bbcc0-01d0-4d09-b4ce-4f82d15c6e07 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:54.250240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:54.269257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:54.287083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb718b-3fd2-4fa4-b907-6e262cf52a7a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:54.329378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b7726-7dd7-41cb-9634-3c6b3fc067da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:54.330103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b7726-7dd7-41cb-9634-3c6b3fc067da None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:54.330433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b7726-7dd7-41cb-9634-3c6b3fc067da None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:54.331160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b7726-7dd7-41cb-9634-3c6b3fc067da None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:54.331455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b7726-7dd7-41cb-9634-3c6b3fc067da None admin] RBAC: Adding query filter params (name=reader) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:54.331764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b7726-7dd7-41cb-9634-3c6b3fc067da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:54.333179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b7726-7dd7-41cb-9634-3c6b3fc067da None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:54.344564 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 180/359] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:54.345894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b7726-7dd7-41cb-9634-3c6b3fc067da None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:54.389677 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bbcc0-01d0-4d09-b4ce-4f82d15c6e07 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:54.429890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5602c-0b78-4b91-8e04-0f46f29fc068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:54.430923 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5602c-0b78-4b91-8e04-0f46f29fc068 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:54.431211 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5602c-0b78-4b91-8e04-0f46f29fc068 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:54.431489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5602c-0b78-4b91-8e04-0f46f29fc068 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:54.432251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5602c-0b78-4b91-8e04-0f46f29fc068 None admin] RBAC: Adding query filter params (name=member) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:54.432565 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5602c-0b78-4b91-8e04-0f46f29fc068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:54.433971 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5602c-0b78-4b91-8e04-0f46f29fc068 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:54.445160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 180/360] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:54.447177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6f703-8c8d-4e76-a744-1af9c8394804 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:54.447476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6f703-8c8d-4e76-a744-1af9c8394804 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:54.447757 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6f703-8c8d-4e76-a744-1af9c8394804 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:54.449118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 181/361] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:54.449939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6f703-8c8d-4e76-a744-1af9c8394804 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:54.484427 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b7726-7dd7-41cb-9634-3c6b3fc067da None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:54.536036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49d093-8b06-4f8f-8d02-4473a5c427ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:54.536786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49d093-8b06-4f8f-8d02-4473a5c427ba None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:54.537123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49d093-8b06-4f8f-8d02-4473a5c427ba None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:54.537402 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49d093-8b06-4f8f-8d02-4473a5c427ba None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:54.538166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49d093-8b06-4f8f-8d02-4473a5c427ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:54.538471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49d093-8b06-4f8f-8d02-4473a5c427ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:54.538586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:54.539962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49d093-8b06-4f8f-8d02-4473a5c427ba None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:54.556341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 181/362] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:44:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:54.557341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49d093-8b06-4f8f-8d02-4473a5c427ba None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:54.571824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:54.585415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6f703-8c8d-4e76-a744-1af9c8394804 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:54.623506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71672baf-0afa-472c-8d6e-c8028f25363f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:54.624174 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71672baf-0afa-472c-8d6e-c8028f25363f None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:54.624462 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71672baf-0afa-472c-8d6e-c8028f25363f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:54.627185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71672baf-0afa-472c-8d6e-c8028f25363f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:54.627185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71672baf-0afa-472c-8d6e-c8028f25363f None admin] RBAC: Adding query filter params (group.id=8327f367eb2848039fa0c9e8eae691d0, role.id=f7f18826d4b44c1284ac31e25088afbd, scope.project.id=c65d427580b743dba5cb657edb44cbbc) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:54.627185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71672baf-0afa-472c-8d6e-c8028f25363f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:54.629248 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71672baf-0afa-472c-8d6e-c8028f25363f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:54.635647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 182/363] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 09:44:54 2022] GET /identity/v3/role_assignments?group.id=8327f367eb2848039fa0c9e8eae691d0&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:54.636485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71672baf-0afa-472c-8d6e-c8028f25363f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:54.695979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49d093-8b06-4f8f-8d02-4473a5c427ba None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:54.749364 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291bfc9-a444-4906-a188-d6b03e0d65bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:54.751250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291bfc9-a444-4906-a188-d6b03e0d65bb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:54.751718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291bfc9-a444-4906-a188-d6b03e0d65bb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:54.752398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291bfc9-a444-4906-a188-d6b03e0d65bb None admin] PATH_INFO: `/v3/users/8c283fb281d84634a5a03349ce100729` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:54.754720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291bfc9-a444-4906-a188-d6b03e0d65bb None admin] RBAC: Authorizing `identity:get_user(user_id=8c283fb281d84634a5a03349ce100729)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:54.756708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291bfc9-a444-4906-a188-d6b03e0d65bb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:54.759314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 182/364] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:44:54 2022] GET /identity/v3/users/8c283fb281d84634a5a03349ce100729 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:54.760993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264be0f-2aeb-4c52-803d-e3d97ac51da2 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:54.761476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264be0f-2aeb-4c52-803d-e3d97ac51da2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:54.762446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264be0f-2aeb-4c52-803d-e3d97ac51da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:54.776758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71672baf-0afa-472c-8d6e-c8028f25363f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:54.786254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3264be0f-2aeb-4c52-803d-e3d97ac51da2 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:54.788576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3264be0f-2aeb-4c52-803d-e3d97ac51da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:54.816152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:54.825289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79301765-6747-4039-bcca-13ba75d94f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:54.826416 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79301765-6747-4039-bcca-13ba75d94f1c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:54.826922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79301765-6747-4039-bcca-13ba75d94f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:54.827419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79301765-6747-4039-bcca-13ba75d94f1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:54.876294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 183/365] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:54.877553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264be0f-2aeb-4c52-803d-e3d97ac51da2 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:54.890735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:54.914538 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79301765-6747-4039-bcca-13ba75d94f1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:54.916429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79301765-6747-4039-bcca-13ba75d94f1c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:55.012585 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264be0f-2aeb-4c52-803d-e3d97ac51da2 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:55.035287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 183/366] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:55.036324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79301765-6747-4039-bcca-13ba75d94f1c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:55.064398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d503b5a-1fab-4448-a663-30def0179f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:55.065529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d503b5a-1fab-4448-a663-30def0179f3a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:55.066259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d503b5a-1fab-4448-a663-30def0179f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:55.066753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d503b5a-1fab-4448-a663-30def0179f3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:55.076934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:55.172775 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d503b5a-1fab-4448-a663-30def0179f3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:55.175664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d503b5a-1fab-4448-a663-30def0179f3a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:55.191996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:55.199919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79301765-6747-4039-bcca-13ba75d94f1c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:55.242376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d98dda-a634-490d-b45b-2e3c9caa04a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:55.243084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d98dda-a634-490d-b45b-2e3c9caa04a9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:55.243363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d98dda-a634-490d-b45b-2e3c9caa04a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:55.243649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d98dda-a634-490d-b45b-2e3c9caa04a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:55.244484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d98dda-a634-490d-b45b-2e3c9caa04a9 None admin] RBAC: Adding query filter params (role.id=61240f79e73148dbbfc847f44c528190, user.id=8c283fb281d84634a5a03349ce100729, scope.system=all) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:55.244802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d98dda-a634-490d-b45b-2e3c9caa04a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:55.246262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d98dda-a634-490d-b45b-2e3c9caa04a9 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:55.259844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 184/367] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jun 29 09:44:55 2022] GET /identity/v3/role_assignments?user.id=8c283fb281d84634a5a03349ce100729&scope.system=all&role.id=61240f79e73148dbbfc847f44c528190 => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:55.260771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d98dda-a634-490d-b45b-2e3c9caa04a9 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:55.349475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 184/368] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:55.350658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d503b5a-1fab-4448-a663-30def0179f3a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:55.440943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:55.467043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d98dda-a634-490d-b45b-2e3c9caa04a9 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:55.528678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864eaccb-f9cb-4d8f-82de-744daff156f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:55.529509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864eaccb-f9cb-4d8f-82de-744daff156f1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:55.529795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864eaccb-f9cb-4d8f-82de-744daff156f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:55.530072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864eaccb-f9cb-4d8f-82de-744daff156f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:55.544342 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d503b5a-1fab-4448-a663-30def0179f3a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:55.546855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864eaccb-f9cb-4d8f-82de-744daff156f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:55.552936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-864eaccb-f9cb-4d8f-82de-744daff156f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:55.555155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:55.556504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:55.557281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 185/369] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:44:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:55.557916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864eaccb-f9cb-4d8f-82de-744daff156f1 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:55.601377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac92905-f0de-49d1-bce3-17e38b29c1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:55.602443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac92905-f0de-49d1-bce3-17e38b29c1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:55.602799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac92905-f0de-49d1-bce3-17e38b29c1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:55.604014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac92905-f0de-49d1-bce3-17e38b29c1d6 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:55.605282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac92905-f0de-49d1-bce3-17e38b29c1d6 None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:55.606809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac92905-f0de-49d1-bce3-17e38b29c1d6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:55.609465 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 185/370] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:55 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:55.610757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac92905-f0de-49d1-bce3-17e38b29c1d6 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:55.688153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864eaccb-f9cb-4d8f-82de-744daff156f1 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:55.746236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ea082-2e9a-4111-89c4-253ac214ff06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:55.748172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ea082-2e9a-4111-89c4-253ac214ff06 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:55.748647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ea082-2e9a-4111-89c4-253ac214ff06 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:55.749123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ea082-2e9a-4111-89c4-253ac214ff06 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:55.750636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ea082-2e9a-4111-89c4-253ac214ff06 None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:55.753972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ea082-2e9a-4111-89c4-253ac214ff06 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:55.757491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 186/371] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:55 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:55.758854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ea082-2e9a-4111-89c4-253ac214ff06 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:55.770301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac92905-f0de-49d1-bce3-17e38b29c1d6 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:55.786140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:55.819875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add9f25-e21d-4abf-90a2-600085d2963d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:55.820585 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add9f25-e21d-4abf-90a2-600085d2963d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:55.821276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add9f25-e21d-4abf-90a2-600085d2963d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:55.821563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add9f25-e21d-4abf-90a2-600085d2963d None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:55.822653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add9f25-e21d-4abf-90a2-600085d2963d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:55.822976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add9f25-e21d-4abf-90a2-600085d2963d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:55.825160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add9f25-e21d-4abf-90a2-600085d2963d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:55.838523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 186/372] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:55.839446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add9f25-e21d-4abf-90a2-600085d2963d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:55.888351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:55.897599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ea082-2e9a-4111-89c4-253ac214ff06 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:55.942120 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a213c43-7efe-4ad0-87db-fb9c026a5b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:55.943056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a213c43-7efe-4ad0-87db-fb9c026a5b34 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:55.943340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a213c43-7efe-4ad0-87db-fb9c026a5b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:55.943746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a213c43-7efe-4ad0-87db-fb9c026a5b34 None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:55.947666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a213c43-7efe-4ad0-87db-fb9c026a5b34 None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:55.949149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a213c43-7efe-4ad0-87db-fb9c026a5b34 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:55.951228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 187/373] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:44:55 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:55.952018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a213c43-7efe-4ad0-87db-fb9c026a5b34 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:55.985558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add9f25-e21d-4abf-90a2-600085d2963d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:56.034737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45cf5b-65e3-4867-a775-faa190a03396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:56.035632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45cf5b-65e3-4867-a775-faa190a03396 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:56.035921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45cf5b-65e3-4867-a775-faa190a03396 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:56.036258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45cf5b-65e3-4867-a775-faa190a03396 None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:56.037589 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45cf5b-65e3-4867-a775-faa190a03396 None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:56.039461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45cf5b-65e3-4867-a775-faa190a03396 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:56.041282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 187/374] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:44:55 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:56.042295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45cf5b-65e3-4867-a775-faa190a03396 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:56.061951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:56.088227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a213c43-7efe-4ad0-87db-fb9c026a5b34 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:56.132828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f159ff-7726-4e26-9fbb-9587bb74a75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:56.133747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f159ff-7726-4e26-9fbb-9587bb74a75c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:56.134227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f159ff-7726-4e26-9fbb-9587bb74a75c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:56.134697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f159ff-7726-4e26-9fbb-9587bb74a75c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:56.186874 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45cf5b-65e3-4867-a775-faa190a03396 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:56.190738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:56.230468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f159ff-7726-4e26-9fbb-9587bb74a75c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:56.231084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f159ff-7726-4e26-9fbb-9587bb74a75c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:56.232039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01370348-f62f-46d0-b95c-300b24008ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:56.232956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01370348-f62f-46d0-b95c-300b24008ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:56.233468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01370348-f62f-46d0-b95c-300b24008ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:56.233916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01370348-f62f-46d0-b95c-300b24008ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:56.234979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01370348-f62f-46d0-b95c-300b24008ff0 None admin] RBAC: Adding query filter params (group.id=8327f367eb2848039fa0c9e8eae691d0, role.id=f7f18826d4b44c1284ac31e25088afbd, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:56.235476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01370348-f62f-46d0-b95c-300b24008ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:56.237417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01370348-f62f-46d0-b95c-300b24008ff0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:56.244880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 188/375] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 09:44:56 2022] GET /identity/v3/role_assignments?group.id=8327f367eb2848039fa0c9e8eae691d0&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:56.246130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01370348-f62f-46d0-b95c-300b24008ff0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:56.332144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:56.355344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 188/376] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:44:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:44:56.365109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f159ff-7726-4e26-9fbb-9587bb74a75c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:56.385873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01370348-f62f-46d0-b95c-300b24008ff0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:56.430415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32b9ef-3911-49a9-a419-75839c10b722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:56.431287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32b9ef-3911-49a9-a419-75839c10b722 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:56.431576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32b9ef-3911-49a9-a419-75839c10b722 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:56.431854 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32b9ef-3911-49a9-a419-75839c10b722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:56.433161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32b9ef-3911-49a9-a419-75839c10b722 None admin] RBAC: Adding query filter params (group.id=8327f367eb2848039fa0c9e8eae691d0, role.id=f819d0f29adf41a49fa045dc3c031527, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:56.433484 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32b9ef-3911-49a9-a419-75839c10b722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:56.435351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32b9ef-3911-49a9-a419-75839c10b722 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:56.443916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 189/377] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 09:44:56 2022] GET /identity/v3/role_assignments?group.id=8327f367eb2848039fa0c9e8eae691d0&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=f819d0f29adf41a49fa045dc3c031527 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:56.490110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f159ff-7726-4e26-9fbb-9587bb74a75c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:56.531991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708de374-2c80-4830-b275-21020810dbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:56.533458 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708de374-2c80-4830-b275-21020810dbaf None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:56.533955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708de374-2c80-4830-b275-21020810dbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:56.534438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708de374-2c80-4830-b275-21020810dbaf None admin] PATH_INFO: `/v3/projects/d79cb8b2f46e418d955fc3be77ed8c7a` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:56.536045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708de374-2c80-4830-b275-21020810dbaf None admin] RBAC: Authorizing `identity:get_project(project_id=d79cb8b2f46e418d955fc3be77ed8c7a)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:56.537895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708de374-2c80-4830-b275-21020810dbaf None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:56.544913 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 189/378] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:56 2022] GET /identity/v3/projects/d79cb8b2f46e418d955fc3be77ed8c7a => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:56.546824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32b9ef-3911-49a9-a419-75839c10b722 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:56.563883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515d50f-daff-460d-ae4c-f22099d65f33 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:56.564628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515d50f-daff-460d-ae4c-f22099d65f33 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:56.565013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515d50f-daff-460d-ae4c-f22099d65f33 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:56.566390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 190/379] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:56.572474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24774c8-57ef-4004-84f0-48d729e4e182 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:56.572781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24774c8-57ef-4004-84f0-48d729e4e182 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:56.573048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24774c8-57ef-4004-84f0-48d729e4e182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:56.596311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-d24774c8-57ef-4004-84f0-48d729e4e182 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:56.597670 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d24774c8-57ef-4004-84f0-48d729e4e182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:56.600636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:56.672894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:56.700918 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 191/380] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:56.724081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32b9ef-3911-49a9-a419-75839c10b722 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:56.728567 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24774c8-57ef-4004-84f0-48d729e4e182 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:56.762866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21555b17-e7b6-4524-bf45-27aa1329abfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:56.763758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21555b17-e7b6-4524-bf45-27aa1329abfc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:56.764035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21555b17-e7b6-4524-bf45-27aa1329abfc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:56.764306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21555b17-e7b6-4524-bf45-27aa1329abfc None admin] PATH_INFO: `/v3/groups/ca38cf14c3b2434ab7b496ada0049811` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:56.772646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21555b17-e7b6-4524-bf45-27aa1329abfc None admin] RBAC: Authorizing `identity:get_group(group_id=ca38cf14c3b2434ab7b496ada0049811)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:56.774140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21555b17-e7b6-4524-bf45-27aa1329abfc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:56.776426 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 190/381] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:44:56 2022] GET /identity/v3/groups/ca38cf14c3b2434ab7b496ada0049811 => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:56.781867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21555b17-e7b6-4524-bf45-27aa1329abfc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:56.932603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:56.938534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21555b17-e7b6-4524-bf45-27aa1329abfc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:56.949456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24774c8-57ef-4004-84f0-48d729e4e182 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:56.977718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db2e1a-916e-4b57-b6c6-c9c0a4ecc04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:56.978369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db2e1a-916e-4b57-b6c6-c9c0a4ecc04b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:56.978681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db2e1a-916e-4b57-b6c6-c9c0a4ecc04b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:56.978961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db2e1a-916e-4b57-b6c6-c9c0a4ecc04b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:56.979889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db2e1a-916e-4b57-b6c6-c9c0a4ecc04b None admin] RBAC: Adding query filter params (group.id=ca38cf14c3b2434ab7b496ada0049811, role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=d79cb8b2f46e418d955fc3be77ed8c7a) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:56.980205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db2e1a-916e-4b57-b6c6-c9c0a4ecc04b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:56.981676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db2e1a-916e-4b57-b6c6-c9c0a4ecc04b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:56.988579 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 191/382] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 09:44:56 2022] GET /identity/v3/role_assignments?group.id=ca38cf14c3b2434ab7b496ada0049811&scope.project.id=d79cb8b2f46e418d955fc3be77ed8c7a&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:57.032288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a6c67-2f96-4cc5-a3cf-e80c210013f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:57.033235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6c67-2f96-4cc5-a3cf-e80c210013f0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:57.033522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6c67-2f96-4cc5-a3cf-e80c210013f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:57.033800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6c67-2f96-4cc5-a3cf-e80c210013f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:57.052649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a6c67-2f96-4cc5-a3cf-e80c210013f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-539a6c67-2f96-4cc5-a3cf-e80c210013f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:57.061228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:57.062167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:57.063031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 192/383] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:44:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:57.068908 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db2e1a-916e-4b57-b6c6-c9c0a4ecc04b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:57.212687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db2e1a-916e-4b57-b6c6-c9c0a4ecc04b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:57.262271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e0f7f-09bc-4167-a4aa-e99e41e6dc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:57.262933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e0f7f-09bc-4167-a4aa-e99e41e6dc26 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:57.263218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e0f7f-09bc-4167-a4aa-e99e41e6dc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:57.263495 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e0f7f-09bc-4167-a4aa-e99e41e6dc26 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:57.264275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e0f7f-09bc-4167-a4aa-e99e41e6dc26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:57.264587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e0f7f-09bc-4167-a4aa-e99e41e6dc26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:57.266001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e0f7f-09bc-4167-a4aa-e99e41e6dc26 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:57.278756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 192/384] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:44:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:57.283961 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a6c67-2f96-4cc5-a3cf-e80c210013f0 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:57.356284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:57.434345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a6c67-2f96-4cc5-a3cf-e80c210013f0 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:57.464136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3b2cd-4a44-4785-9d90-b74f0b339ab1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:57.464437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3b2cd-4a44-4785-9d90-b74f0b339ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:57.464716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3b2cd-4a44-4785-9d90-b74f0b339ab1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:57.467767 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 193/385] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:57.472123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb97274-c30c-4068-89e5-f180f32c4450 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:57.472417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb97274-c30c-4068-89e5-f180f32c4450 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:57.472698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb97274-c30c-4068-89e5-f180f32c4450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:57.472799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5838e9-ec16-479f-94e2-0f09c8a6fce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:57.473849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5838e9-ec16-479f-94e2-0f09c8a6fce4 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:57.474308 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5838e9-ec16-479f-94e2-0f09c8a6fce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:57.475134 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5838e9-ec16-479f-94e2-0f09c8a6fce4 None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:57.476510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5838e9-ec16-479f-94e2-0f09c8a6fce4 None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:57.478147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5838e9-ec16-479f-94e2-0f09c8a6fce4 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:57.480367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 193/386] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:44:57 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:57.487209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5838e9-ec16-479f-94e2-0f09c8a6fce4 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:57.506273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6fb97274-c30c-4068-89e5-f180f32c4450 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:57.507621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6fb97274-c30c-4068-89e5-f180f32c4450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:57.531122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:57.631477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 194/387] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:57.632426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5838e9-ec16-479f-94e2-0f09c8a6fce4 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:57.641758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:57.665272 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb97274-c30c-4068-89e5-f180f32c4450 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:57.669614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be2acb-dff8-4507-917f-6ff3e9ab5e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:57.670512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be2acb-dff8-4507-917f-6ff3e9ab5e5f None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:57.670827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be2acb-dff8-4507-917f-6ff3e9ab5e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:57.671104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be2acb-dff8-4507-917f-6ff3e9ab5e5f None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:57.672349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be2acb-dff8-4507-917f-6ff3e9ab5e5f None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:57.674031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be2acb-dff8-4507-917f-6ff3e9ab5e5f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:57.676480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 194/388] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:57 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:57.682542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be2acb-dff8-4507-917f-6ff3e9ab5e5f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:57.807503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb97274-c30c-4068-89e5-f180f32c4450 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:57.848199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4ca0a-0755-4dca-acfd-4f039249a014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:57.849048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4ca0a-0755-4dca-acfd-4f039249a014 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:57.849328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4ca0a-0755-4dca-acfd-4f039249a014 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:57.849951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4ca0a-0755-4dca-acfd-4f039249a014 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:57.862088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4ca0a-0755-4dca-acfd-4f039249a014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-bfe4ca0a-0755-4dca-acfd-4f039249a014 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:57.870391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:57.871333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:57.872114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 195/389] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:44:57.877379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4ca0a-0755-4dca-acfd-4f039249a014 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:57.895745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:57.976303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be2acb-dff8-4507-917f-6ff3e9ab5e5f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:57.998241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4ca0a-0755-4dca-acfd-4f039249a014 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:58.045150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708471f-70c5-4631-9d5f-7f9adaa94a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:58.046008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708471f-70c5-4631-9d5f-7f9adaa94a29 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:58.046388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708471f-70c5-4631-9d5f-7f9adaa94a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:58.046797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708471f-70c5-4631-9d5f-7f9adaa94a29 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:58.047819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708471f-70c5-4631-9d5f-7f9adaa94a29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:58.048229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708471f-70c5-4631-9d5f-7f9adaa94a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:58.052719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708471f-70c5-4631-9d5f-7f9adaa94a29 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:58.062941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 196/390] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:58.068939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708471f-70c5-4631-9d5f-7f9adaa94a29 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:58.082749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1247-a94c-4cc6-bf6c-2b2f04ff7d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:58.083771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1247-a94c-4cc6-bf6c-2b2f04ff7d64 None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:58.084062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1247-a94c-4cc6-bf6c-2b2f04ff7d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:58.084338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1247-a94c-4cc6-bf6c-2b2f04ff7d64 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc/groups/8327f367eb2848039fa0c9e8eae691d0/roles/f819d0f29adf41a49fa045dc3c031527` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:58.095398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca1247-a94c-4cc6-bf6c-2b2f04ff7d64 None admin] RBAC: Authorizing `identity:create_grant(project_id=c65d427580b743dba5cb657edb44cbbc, group_id=8327f367eb2848039fa0c9e8eae691d0, role_id=f819d0f29adf41a49fa045dc3c031527)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:58.097064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca1247-a94c-4cc6-bf6c-2b2f04ff7d64 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:58.138318 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 195/391] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jun 29 09:44:57 2022] PUT /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc/groups/8327f367eb2848039fa0c9e8eae691d0/roles/f819d0f29adf41a49fa045dc3c031527 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:44:58.163127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:58.201265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708471f-70c5-4631-9d5f-7f9adaa94a29 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:58.254206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e01a4e-bf15-4897-951d-d500fa0074e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:58.255123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01a4e-bf15-4897-951d-d500fa0074e2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:58.255401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01a4e-bf15-4897-951d-d500fa0074e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:58.255673 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01a4e-bf15-4897-951d-d500fa0074e2 None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:58.259464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e01a4e-bf15-4897-951d-d500fa0074e2 None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:58.261004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e01a4e-bf15-4897-951d-d500fa0074e2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:58.263228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 197/392] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:44:58 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:58.271838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1247-a94c-4cc6-bf6c-2b2f04ff7d64 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:58.334678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:58.411412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1247-a94c-4cc6-bf6c-2b2f04ff7d64 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:58.448061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960641d8-27aa-4635-bf51-b3ab59cd5cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:58.448951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960641d8-27aa-4635-bf51-b3ab59cd5cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:58.449235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960641d8-27aa-4635-bf51-b3ab59cd5cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:58.449513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960641d8-27aa-4635-bf51-b3ab59cd5cd7 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:58.450785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960641d8-27aa-4635-bf51-b3ab59cd5cd7 None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:58.452284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960641d8-27aa-4635-bf51-b3ab59cd5cd7 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:58.454393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 196/393] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:58 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:58.461257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e01a4e-bf15-4897-951d-d500fa0074e2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:58.575367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e01a4e-bf15-4897-951d-d500fa0074e2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:58.627044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8d041-6523-4095-bdc4-d3095e60b0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:58.628956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8d041-6523-4095-bdc4-d3095e60b0b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:58.629470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8d041-6523-4095-bdc4-d3095e60b0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:58.629939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8d041-6523-4095-bdc4-d3095e60b0b8 None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc/groups/8327f367eb2848039fa0c9e8eae691d0/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:58.632168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8d041-6523-4095-bdc4-d3095e60b0b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c65d427580b743dba5cb657edb44cbbc, group_id=8327f367eb2848039fa0c9e8eae691d0, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:58.633844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8d041-6523-4095-bdc4-d3095e60b0b8 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:58.663735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 198/394] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jun 29 09:44:58 2022] PUT /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc/groups/8327f367eb2848039fa0c9e8eae691d0/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:44:58.875966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88702e45-308c-42a8-af6e-235eb2383661 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:58.876589 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88702e45-308c-42a8-af6e-235eb2383661 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:58.877234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88702e45-308c-42a8-af6e-235eb2383661 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:58.879470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 197/395] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:58.884962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64ee96-d49f-4709-9649-b7c83797903c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:58.885500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64ee96-d49f-4709-9649-b7c83797903c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:58.885973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64ee96-d49f-4709-9649-b7c83797903c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:58.891589 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:58.909877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-bb64ee96-d49f-4709-9649-b7c83797903c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:58.911610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bb64ee96-d49f-4709-9649-b7c83797903c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:58.963354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86cbbe-ef3a-43a3-8609-c56f49d11efe None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:58.963976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86cbbe-ef3a-43a3-8609-c56f49d11efe None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:58.964447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86cbbe-ef3a-43a3-8609-c56f49d11efe None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:58.966109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 198/396] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:58.973344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006e497-74ce-4a6f-8b7e-ce3355c77d96 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:58.974003 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006e497-74ce-4a6f-8b7e-ce3355c77d96 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:58.974370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006e497-74ce-4a6f-8b7e-ce3355c77d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:58.996922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-4006e497-74ce-4a6f-8b7e-ce3355c77d96 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:58.999067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4006e497-74ce-4a6f-8b7e-ce3355c77d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:59.002043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 199/397] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:59.034829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64ee96-d49f-4709-9649-b7c83797903c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:59.141640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 199/398] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:44:59.203886 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006e497-74ce-4a6f-8b7e-ce3355c77d96 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:59.232827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64ee96-d49f-4709-9649-b7c83797903c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:59.245873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:59.291572 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:59.300283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e3f41-ab0c-4bd4-af41-15b72baebaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:59.301112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e3f41-ab0c-4bd4-af41-15b72baebaac None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:59.301398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e3f41-ab0c-4bd4-af41-15b72baebaac None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:59.301670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e3f41-ab0c-4bd4-af41-15b72baebaac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:59.323402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006e497-74ce-4a6f-8b7e-ce3355c77d96 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:59.335578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e3f41-ab0c-4bd4-af41-15b72baebaac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-667e3f41-ab0c-4bd4-af41-15b72baebaac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:59.343538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:59.344688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:59.345912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 200/399] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:44:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:59.351103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e3f41-ab0c-4bd4-af41-15b72baebaac None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:59.366196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301007b-6252-47da-805c-cc287d31ea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:59.367812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301007b-6252-47da-805c-cc287d31ea04 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:59.368235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301007b-6252-47da-805c-cc287d31ea04 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:59.368648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301007b-6252-47da-805c-cc287d31ea04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:59.380657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301007b-6252-47da-805c-cc287d31ea04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-4301007b-6252-47da-805c-cc287d31ea04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:59.388922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:59.389828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:44:59.391148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 200/400] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:44:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:59.398740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301007b-6252-47da-805c-cc287d31ea04 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:59.536755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e3f41-ab0c-4bd4-af41-15b72baebaac None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:59.581421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb358145-f368-48ca-a4e6-0efdfc588322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:59.582109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb358145-f368-48ca-a4e6-0efdfc588322 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:59.582393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb358145-f368-48ca-a4e6-0efdfc588322 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:59.582695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb358145-f368-48ca-a4e6-0efdfc588322 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:59.583469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb358145-f368-48ca-a4e6-0efdfc588322 None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:59.585409 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:59.588066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb358145-f368-48ca-a4e6-0efdfc588322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:59.589547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb358145-f368-48ca-a4e6-0efdfc588322 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:59.593401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301007b-6252-47da-805c-cc287d31ea04 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:59.599070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 201/401] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:44:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:59.600539 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99adf23-8f56-4bb4-8601-559782761fe6 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:59.600820 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99adf23-8f56-4bb4-8601-559782761fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:59.601091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99adf23-8f56-4bb4-8601-559782761fe6 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:59.606923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 202/402] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:44:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:44:59.607772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99adf23-8f56-4bb4-8601-559782761fe6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:59.631872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560c9e0-ab2b-4584-9608-3519b1faf770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:59.632662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560c9e0-ab2b-4584-9608-3519b1faf770 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:59.632941 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560c9e0-ab2b-4584-9608-3519b1faf770 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:59.633213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560c9e0-ab2b-4584-9608-3519b1faf770 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:59.634076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560c9e0-ab2b-4584-9608-3519b1faf770 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:44:59.634380 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560c9e0-ab2b-4584-9608-3519b1faf770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:59.635806 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560c9e0-ab2b-4584-9608-3519b1faf770 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:59.645797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 201/403] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:44:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:59.647379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fe760-0b7e-468f-83fb-eb28018b1d6f None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:59.647664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fe760-0b7e-468f-83fb-eb28018b1d6f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:59.647946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fe760-0b7e-468f-83fb-eb28018b1d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:59.667219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b07fe760-0b7e-468f-83fb-eb28018b1d6f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:44:59.668326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b07fe760-0b7e-468f-83fb-eb28018b1d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:44:59.676102 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:59.728359 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99adf23-8f56-4bb4-8601-559782761fe6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:59.750827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 202/404] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:44:59.751778 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fe760-0b7e-468f-83fb-eb28018b1d6f None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:59.778944 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40249b0-e2b6-4831-9d90-4f241d5729c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:59.780119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40249b0-e2b6-4831-9d90-4f241d5729c5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:59.780565 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40249b0-e2b6-4831-9d90-4f241d5729c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:59.781004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40249b0-e2b6-4831-9d90-4f241d5729c5 None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:59.782468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40249b0-e2b6-4831-9d90-4f241d5729c5 None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:59.784847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40249b0-e2b6-4831-9d90-4f241d5729c5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:59.786705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 203/405] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:44:59 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:59.787645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40249b0-e2b6-4831-9d90-4f241d5729c5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:59.859834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:59.875786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fe760-0b7e-468f-83fb-eb28018b1d6f None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:59.899498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40249b0-e2b6-4831-9d90-4f241d5729c5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:44:59.912027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6c40c-c427-4fd3-974c-0c1a5890b8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:59.912987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6c40c-c427-4fd3-974c-0c1a5890b8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:59.913266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6c40c-c427-4fd3-974c-0c1a5890b8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:59.913536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6c40c-c427-4fd3-974c-0c1a5890b8b2 None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:59.914772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6c40c-c427-4fd3-974c-0c1a5890b8b2 None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:44:59.916325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6c40c-c427-4fd3-974c-0c1a5890b8b2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:44:59.918277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 203/406] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:44:59 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:44:59.919015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6c40c-c427-4fd3-974c-0c1a5890b8b2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:44:59.927682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:44:59.941543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9935b2-61e9-4339-b3d3-cb439e5ae2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:44:59.942669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9935b2-61e9-4339-b3d3-cb439e5ae2cc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:44:59.943138 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9935b2-61e9-4339-b3d3-cb439e5ae2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:44:59.943646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9935b2-61e9-4339-b3d3-cb439e5ae2cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:44:59.955328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9935b2-61e9-4339-b3d3-cb439e5ae2cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-4b9935b2-61e9-4339-b3d3-cb439e5ae2cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:44:59.963949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:44:59.964809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:44:59.965611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:44:59.968033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 204/407] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:44:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:44:59.975020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9935b2-61e9-4339-b3d3-cb439e5ae2cc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:00.028113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6c40c-c427-4fd3-974c-0c1a5890b8b2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:00.067808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b10db-2cc8-4f06-9959-5b26dce9f8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:00.068689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b10db-2cc8-4f06-9959-5b26dce9f8ef None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:00.068965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b10db-2cc8-4f06-9959-5b26dce9f8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:00.069234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b10db-2cc8-4f06-9959-5b26dce9f8ef None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:00.070532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b10db-2cc8-4f06-9959-5b26dce9f8ef None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:00.072094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b10db-2cc8-4f06-9959-5b26dce9f8ef None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:00.073989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 204/408] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:59 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:00.079694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b10db-2cc8-4f06-9959-5b26dce9f8ef None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:00.091776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9935b2-61e9-4339-b3d3-cb439e5ae2cc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:00.107323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:00.133078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a66c-2b6a-46fb-bbee-9d267e04abd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:00.134343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a66c-2b6a-46fb-bbee-9d267e04abd7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:00.134846 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a66c-2b6a-46fb-bbee-9d267e04abd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:00.135334 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a66c-2b6a-46fb-bbee-9d267e04abd7 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:00.136981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4a66c-2b6a-46fb-bbee-9d267e04abd7 None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:00.138785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4a66c-2b6a-46fb-bbee-9d267e04abd7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:00.141389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 205/409] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:44:59 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:00.144666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a66c-2b6a-46fb-bbee-9d267e04abd7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:00.192815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b10db-2cc8-4f06-9959-5b26dce9f8ef None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:00.200942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:00.233449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0346b-e6b8-408d-8695-3ecf87e34a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:00.234270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0346b-e6b8-408d-8695-3ecf87e34a57 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:00.234551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0346b-e6b8-408d-8695-3ecf87e34a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:00.234889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0346b-e6b8-408d-8695-3ecf87e34a57 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:00.235641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0346b-e6b8-408d-8695-3ecf87e34a57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:00.236002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0346b-e6b8-408d-8695-3ecf87e34a57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:00.237509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0346b-e6b8-408d-8695-3ecf87e34a57 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:00.247167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 205/410] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:00.248343 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0346b-e6b8-408d-8695-3ecf87e34a57 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:00.264173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a66c-2b6a-46fb-bbee-9d267e04abd7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:00.301783 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6dd5c0-9b4e-412d-9418-707917373b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:00.303085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dd5c0-9b4e-412d-9418-707917373b9d None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:00.303556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dd5c0-9b4e-412d-9418-707917373b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:00.304021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dd5c0-9b4e-412d-9418-707917373b9d None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/groups/8327f367eb2848039fa0c9e8eae691d0/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:00.306486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6dd5c0-9b4e-412d-9418-707917373b9d None admin] RBAC: Authorizing `identity:create_grant(project_id=ea40190bfcd344248c8e0d7f7860ef36, group_id=8327f367eb2848039fa0c9e8eae691d0, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:00.308167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6dd5c0-9b4e-412d-9418-707917373b9d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:00.323865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 206/411] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jun 29 09:45:00 2022] PUT /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/groups/8327f367eb2848039fa0c9e8eae691d0/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:45:00.325234 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6dd5c0-9b4e-412d-9418-707917373b9d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:00.344521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:00.364159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0346b-e6b8-408d-8695-3ecf87e34a57 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:00.404420 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef30f59-e5d5-4235-b0fc-c45d8c2b6c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:00.405504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef30f59-e5d5-4235-b0fc-c45d8c2b6c6e None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:00.405874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef30f59-e5d5-4235-b0fc-c45d8c2b6c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:00.406242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef30f59-e5d5-4235-b0fc-c45d8c2b6c6e None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/groups/8327f367eb2848039fa0c9e8eae691d0/roles/f819d0f29adf41a49fa045dc3c031527` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:00.409403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef30f59-e5d5-4235-b0fc-c45d8c2b6c6e None admin] RBAC: Authorizing `identity:create_grant(project_id=ea40190bfcd344248c8e0d7f7860ef36, group_id=8327f367eb2848039fa0c9e8eae691d0, role_id=f819d0f29adf41a49fa045dc3c031527)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:00.411151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef30f59-e5d5-4235-b0fc-c45d8c2b6c6e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:00.449330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 206/412] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jun 29 09:45:00 2022] PUT /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36/groups/8327f367eb2848039fa0c9e8eae691d0/roles/f819d0f29adf41a49fa045dc3c031527 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:45:00.451559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6dd5c0-9b4e-412d-9418-707917373b9d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:00.469606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:00.494820 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448b6bb-8061-4137-872c-519a7342a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:00.496024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448b6bb-8061-4137-872c-519a7342a16d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:00.496485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448b6bb-8061-4137-872c-519a7342a16d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:00.496893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448b6bb-8061-4137-872c-519a7342a16d None admin] PATH_INFO: `/v3/groups/ca38cf14c3b2434ab7b496ada0049811` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:00.498473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448b6bb-8061-4137-872c-519a7342a16d None admin] RBAC: Authorizing `identity:get_group(group_id=ca38cf14c3b2434ab7b496ada0049811)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:00.500220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448b6bb-8061-4137-872c-519a7342a16d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:00.506717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 207/413] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:45:00 2022] GET /identity/v3/groups/ca38cf14c3b2434ab7b496ada0049811 => generated 229 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:00.511796 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef30f59-e5d5-4235-b0fc-c45d8c2b6c6e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:00.672100 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef30f59-e5d5-4235-b0fc-c45d8c2b6c6e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:00.705276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753fcb1-dd85-4406-b69c-47ea5a9b9a2a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:00.705550 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753fcb1-dd85-4406-b69c-47ea5a9b9a2a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:00.705827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753fcb1-dd85-4406-b69c-47ea5a9b9a2a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:00.707657 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 208/414] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:45:00.713135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b139d7-c329-4cdc-a186-8990d746308f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:00.713425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b139d7-c329-4cdc-a186-8990d746308f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:00.713701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b139d7-c329-4cdc-a186-8990d746308f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:00.716005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2a381-97af-4794-901f-3c9a48fafaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:00.717169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2a381-97af-4794-901f-3c9a48fafaba None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:00.717705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2a381-97af-4794-901f-3c9a48fafaba None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:00.718199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2a381-97af-4794-901f-3c9a48fafaba None admin] PATH_INFO: `/v3/projects/d79cb8b2f46e418d955fc3be77ed8c7a` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:00.719731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2a381-97af-4794-901f-3c9a48fafaba None admin] RBAC: Authorizing `identity:get_project(project_id=d79cb8b2f46e418d955fc3be77ed8c7a)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:00.722091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2a381-97af-4794-901f-3c9a48fafaba None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:00.724912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 207/415] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:45:00 2022] GET /identity/v3/projects/d79cb8b2f46e418d955fc3be77ed8c7a => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:00.731202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2a381-97af-4794-901f-3c9a48fafaba None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:00.733283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d0b139d7-c329-4cdc-a186-8990d746308f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:00.734457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d0b139d7-c329-4cdc-a186-8990d746308f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:00.751954 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:00.809509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 209/416] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:00.833727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b139d7-c329-4cdc-a186-8990d746308f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:00.867729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2a381-97af-4794-901f-3c9a48fafaba None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:00.920391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e929bf-10d7-4371-9b0d-32726158bde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:00.920786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e929bf-10d7-4371-9b0d-32726158bde0 None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:00.922610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e929bf-10d7-4371-9b0d-32726158bde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:00.922610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e929bf-10d7-4371-9b0d-32726158bde0 None admin] PATH_INFO: `/v3/projects/d79cb8b2f46e418d955fc3be77ed8c7a/groups/ca38cf14c3b2434ab7b496ada0049811/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:00.926589 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e929bf-10d7-4371-9b0d-32726158bde0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d79cb8b2f46e418d955fc3be77ed8c7a, group_id=ca38cf14c3b2434ab7b496ada0049811, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:00.928076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e929bf-10d7-4371-9b0d-32726158bde0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:00.953918 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 208/417] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jun 29 09:45:00 2022] PUT /identity/v3/projects/d79cb8b2f46e418d955fc3be77ed8c7a/groups/ca38cf14c3b2434ab7b496ada0049811/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:45:00.961085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b139d7-c329-4cdc-a186-8990d746308f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:00.972145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:01.012445 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c03fc-9b81-4a1d-a242-f899ec75dc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:01.013279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c03fc-9b81-4a1d-a242-f899ec75dc01 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:01.013556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c03fc-9b81-4a1d-a242-f899ec75dc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:01.013828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c03fc-9b81-4a1d-a242-f899ec75dc01 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:01.031290 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c03fc-9b81-4a1d-a242-f899ec75dc01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-351c03fc-9b81-4a1d-a242-f899ec75dc01 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:01.039796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:01.040768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:01.041594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 210/418] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:01.048934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e929bf-10d7-4371-9b0d-32726158bde0 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:01.144689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:01.162998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f207a44-a24b-4fa0-84f8-689e966ce9e1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:01.163313 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f207a44-a24b-4fa0-84f8-689e966ce9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:01.163549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f207a44-a24b-4fa0-84f8-689e966ce9e1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:01.165033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 211/419] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:45:01.170480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274165d-bfd6-42d2-8bf5-7813bd2f6bba None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:01.170799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274165d-bfd6-42d2-8bf5-7813bd2f6bba None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:01.171073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274165d-bfd6-42d2-8bf5-7813bd2f6bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:01.206636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f274165d-bfd6-42d2-8bf5-7813bd2f6bba None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:01.207853 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f274165d-bfd6-42d2-8bf5-7813bd2f6bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:01.210487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e929bf-10d7-4371-9b0d-32726158bde0 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:01.270080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d6245-a3d6-4234-8bfc-0ecf153cdc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:01.271354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d6245-a3d6-4234-8bfc-0ecf153cdc9d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:01.271649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d6245-a3d6-4234-8bfc-0ecf153cdc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:01.271929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d6245-a3d6-4234-8bfc-0ecf153cdc9d None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:01.272704 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d6245-a3d6-4234-8bfc-0ecf153cdc9d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:01.273015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d6245-a3d6-4234-8bfc-0ecf153cdc9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:01.274408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d6245-a3d6-4234-8bfc-0ecf153cdc9d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:01.285676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 209/420] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:01.290694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d6245-a3d6-4234-8bfc-0ecf153cdc9d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:01.321071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 212/421] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 29 09:45:01.350455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274165d-bfd6-42d2-8bf5-7813bd2f6bba None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:01.451908 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d6245-a3d6-4234-8bfc-0ecf153cdc9d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:01.522386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:01.523989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf74128-39de-4b0c-be4c-ba28d8b68f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:01.524660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf74128-39de-4b0c-be4c-ba28d8b68f1c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:01.524945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf74128-39de-4b0c-be4c-ba28d8b68f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:01.525243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf74128-39de-4b0c-be4c-ba28d8b68f1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:01.527848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274165d-bfd6-42d2-8bf5-7813bd2f6bba None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:01.531709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:01.600015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af447994-7b73-4275-91b9-88278c379c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:01.600843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af447994-7b73-4275-91b9-88278c379c61 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:01.601124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af447994-7b73-4275-91b9-88278c379c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:01.601401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af447994-7b73-4275-91b9-88278c379c61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:01.626667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af447994-7b73-4275-91b9-88278c379c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-af447994-7b73-4275-91b9-88278c379c61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:01.635658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:01.636547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:45:01.637410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:01.637997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 213/422] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:01.649978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af447994-7b73-4275-91b9-88278c379c61 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:01.669853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf74128-39de-4b0c-be4c-ba28d8b68f1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:01.671397 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf74128-39de-4b0c-be4c-ba28d8b68f1c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:01.825315 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 210/423] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 09:45:01.831331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf74128-39de-4b0c-be4c-ba28d8b68f1c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:01.874766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af447994-7b73-4275-91b9-88278c379c61 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:01.902138 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:01.951302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467f5fd-e675-4eea-9089-c9e827ec43cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:01.952391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467f5fd-e675-4eea-9089-c9e827ec43cb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:01.952869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467f5fd-e675-4eea-9089-c9e827ec43cb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:01.953338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467f5fd-e675-4eea-9089-c9e827ec43cb None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:01.954337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467f5fd-e675-4eea-9089-c9e827ec43cb None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:01.954898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467f5fd-e675-4eea-9089-c9e827ec43cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:01.956561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467f5fd-e675-4eea-9089-c9e827ec43cb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:01.972328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 214/424] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:45:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:01.973554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467f5fd-e675-4eea-9089-c9e827ec43cb None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:01.994988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf74128-39de-4b0c-be4c-ba28d8b68f1c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:02.018116 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:02.035783 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6611b75-4de0-4eeb-a321-eb712238ee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:02.036682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6611b75-4de0-4eeb-a321-eb712238ee9b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:02.036961 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6611b75-4de0-4eeb-a321-eb712238ee9b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:02.037231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6611b75-4de0-4eeb-a321-eb712238ee9b None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:02.040086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6611b75-4de0-4eeb-a321-eb712238ee9b None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:02.041553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6611b75-4de0-4eeb-a321-eb712238ee9b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:02.044687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 211/425] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:45:01 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:02.050057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6611b75-4de0-4eeb-a321-eb712238ee9b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:02.173280 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467f5fd-e675-4eea-9089-c9e827ec43cb None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:02.240473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442464f8-03a0-4f80-94fb-82e08158c08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:02.241137 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442464f8-03a0-4f80-94fb-82e08158c08c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:02.241418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442464f8-03a0-4f80-94fb-82e08158c08c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:02.241693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442464f8-03a0-4f80-94fb-82e08158c08c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:02.247108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6611b75-4de0-4eeb-a321-eb712238ee9b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:02.250895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:02.318386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc329fb2-9c05-4703-89b8-29f4ec5020c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:02.319296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc329fb2-9c05-4703-89b8-29f4ec5020c6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:02.319575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc329fb2-9c05-4703-89b8-29f4ec5020c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:02.319849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc329fb2-9c05-4703-89b8-29f4ec5020c6 None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:02.321835 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc329fb2-9c05-4703-89b8-29f4ec5020c6 None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:02.323309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc329fb2-9c05-4703-89b8-29f4ec5020c6 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:02.330502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 212/426] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:45:02 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:02.331585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc329fb2-9c05-4703-89b8-29f4ec5020c6 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:02.373544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442464f8-03a0-4f80-94fb-82e08158c08c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:02.375072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442464f8-03a0-4f80-94fb-82e08158c08c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:02.421057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:02.515655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 215/427] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:02.523112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442464f8-03a0-4f80-94fb-82e08158c08c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:02.580544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc329fb2-9c05-4703-89b8-29f4ec5020c6 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:02.659867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1875-c720-4ece-917f-992fa0341a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:02.660939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1875-c720-4ece-917f-992fa0341a01 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:02.661414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1875-c720-4ece-917f-992fa0341a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:02.661876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1875-c720-4ece-917f-992fa0341a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:02.662947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff1875-c720-4ece-917f-992fa0341a01 None admin] RBAC: Adding query filter params (group.id=8327f367eb2848039fa0c9e8eae691d0, role.id=f819d0f29adf41a49fa045dc3c031527, scope.project.id=c65d427580b743dba5cb657edb44cbbc) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:02.663478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff1875-c720-4ece-917f-992fa0341a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:02.665112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff1875-c720-4ece-917f-992fa0341a01 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:02.681363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 213/428] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 09:45:02 2022] GET /identity/v3/role_assignments?group.id=8327f367eb2848039fa0c9e8eae691d0&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=f819d0f29adf41a49fa045dc3c031527 => generated 611 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:02.702363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442464f8-03a0-4f80-94fb-82e08158c08c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:02.727814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:02.744225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3f5eb-8952-4128-aed2-1e7abd150bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:02.745536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3f5eb-8952-4128-aed2-1e7abd150bba None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:02.746021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3f5eb-8952-4128-aed2-1e7abd150bba None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:02.746490 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3f5eb-8952-4128-aed2-1e7abd150bba None admin] PATH_INFO: `/v3/projects/c65d427580b743dba5cb657edb44cbbc` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:02.748126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3f5eb-8952-4128-aed2-1e7abd150bba None admin] RBAC: Authorizing `identity:get_project(project_id=c65d427580b743dba5cb657edb44cbbc)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:02.749857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3f5eb-8952-4128-aed2-1e7abd150bba None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:02.752346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 216/429] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:45:02 2022] GET /identity/v3/projects/c65d427580b743dba5cb657edb44cbbc => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:02.759735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1875-c720-4ece-917f-992fa0341a01 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:02.780044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7facde7-c08c-4e19-8880-814b5d37c22e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:02.780643 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7facde7-c08c-4e19-8880-814b5d37c22e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:02.781155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7facde7-c08c-4e19-8880-814b5d37c22e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:02.783377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 217/430] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:02.789124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55259a9-3523-4572-ab4a-332705b8dbdb None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:02.789725 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55259a9-3523-4572-ab4a-332705b8dbdb None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:02.790193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55259a9-3523-4572-ab4a-332705b8dbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:02.819848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a55259a9-3523-4572-ab4a-332705b8dbdb None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:02.823975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a55259a9-3523-4572-ab4a-332705b8dbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:02.845255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:02.906050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 218/431] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:45:02.931660 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55259a9-3523-4572-ab4a-332705b8dbdb None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:02.935908 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1875-c720-4ece-917f-992fa0341a01 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:02.992023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6545e7-c26d-421d-a181-0891a10e216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:02.993171 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6545e7-c26d-421d-a181-0891a10e216b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:02.993480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6545e7-c26d-421d-a181-0891a10e216b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:02.994124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6545e7-c26d-421d-a181-0891a10e216b None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:02.995366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6545e7-c26d-421d-a181-0891a10e216b None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:02.997362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6545e7-c26d-421d-a181-0891a10e216b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:02.999368 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 214/432] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:45:02 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:03.008233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6545e7-c26d-421d-a181-0891a10e216b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:03.076329 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55259a9-3523-4572-ab4a-332705b8dbdb None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:03.097986 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:03.119180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f5762-62df-4d40-a6e9-790e122c66d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:03.120300 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5762-62df-4d40-a6e9-790e122c66d6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:03.120753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5762-62df-4d40-a6e9-790e122c66d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:03.121241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5762-62df-4d40-a6e9-790e122c66d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:03.132646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f5762-62df-4d40-a6e9-790e122c66d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:03.137105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6545e7-c26d-421d-a181-0891a10e216b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:03.138651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-943f5762-62df-4d40-a6e9-790e122c66d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:03.141538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:03.142793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 219/433] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:03.148139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0796f-04a8-4961-bdee-d6c2d5581d09 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:03.148869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0796f-04a8-4961-bdee-d6c2d5581d09 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:03.148869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0796f-04a8-4961-bdee-d6c2d5581d09 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:03.148869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 220/434] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:03.151111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0796f-04a8-4961-bdee-d6c2d5581d09 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:03.178828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03e723-8288-4b84-99aa-d3d8044caf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:03.179492 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03e723-8288-4b84-99aa-d3d8044caf5c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:03.179770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03e723-8288-4b84-99aa-d3d8044caf5c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:03.180042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03e723-8288-4b84-99aa-d3d8044caf5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:03.180871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03e723-8288-4b84-99aa-d3d8044caf5c None admin] RBAC: Adding query filter params (group.id=8327f367eb2848039fa0c9e8eae691d0, role.id=f7f18826d4b44c1284ac31e25088afbd, scope.project.id=c65d427580b743dba5cb657edb44cbbc) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:03.181185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03e723-8288-4b84-99aa-d3d8044caf5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:03.182659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03e723-8288-4b84-99aa-d3d8044caf5c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:03.188938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 215/435] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 09:45:03 2022] GET /identity/v3/role_assignments?group.id=8327f367eb2848039fa0c9e8eae691d0&scope.project.id=c65d427580b743dba5cb657edb44cbbc&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:03.190436 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47835-d7b7-4e51-87ea-2ec86a7ca25e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:03.190763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47835-d7b7-4e51-87ea-2ec86a7ca25e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:03.191035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47835-d7b7-4e51-87ea-2ec86a7ca25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:03.217771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-19a47835-d7b7-4e51-87ea-2ec86a7ca25e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:03.219079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-19a47835-d7b7-4e51-87ea-2ec86a7ca25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:03.295374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0796f-04a8-4961-bdee-d6c2d5581d09 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:03.298716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 216/436] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:03.323580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a47835-d7b7-4e51-87ea-2ec86a7ca25e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:03.343997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24962925-983f-452d-82c1-7e399ddc884f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:03.344684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24962925-983f-452d-82c1-7e399ddc884f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:03.344960 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24962925-983f-452d-82c1-7e399ddc884f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:03.345231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24962925-983f-452d-82c1-7e399ddc884f None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:03.346583 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24962925-983f-452d-82c1-7e399ddc884f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:03.347592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24962925-983f-452d-82c1-7e399ddc884f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:03.348718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24962925-983f-452d-82c1-7e399ddc884f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:03.363994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 221/437] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:03.371574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24962925-983f-452d-82c1-7e399ddc884f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:03.428927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:03.458191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:03.466013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a47835-d7b7-4e51-87ea-2ec86a7ca25e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:03.513252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24962925-983f-452d-82c1-7e399ddc884f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:03.530599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05381f-eeb6-4859-acdc-cc49aa0880c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:03.531608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05381f-eeb6-4859-acdc-cc49aa0880c3 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:03.531891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05381f-eeb6-4859-acdc-cc49aa0880c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:03.532517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05381f-eeb6-4859-acdc-cc49aa0880c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:03.552524 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05381f-eeb6-4859-acdc-cc49aa0880c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:03.559707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d81657-a44c-42e8-a414-620261b6af4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-4b05381f-eeb6-4859-acdc-cc49aa0880c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:03.560499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:03.561349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 09:45:03.562182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:03.562825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d81657-a44c-42e8-a414-620261b6af4c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:03.562825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d81657-a44c-42e8-a414-620261b6af4c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:03.562825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d81657-a44c-42e8-a414-620261b6af4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:03.563001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 217/438] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:03.564223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7ed00-33a2-477f-a2ad-223924c3ce14 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:03.564465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7ed00-33a2-477f-a2ad-223924c3ce14 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:03.565387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7ed00-33a2-477f-a2ad-223924c3ce14 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:03.568144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 218/439] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:03.574344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7ed00-33a2-477f-a2ad-223924c3ce14 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:03.644749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d81657-a44c-42e8-a414-620261b6af4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:03.646226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d81657-a44c-42e8-a414-620261b6af4c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:03.697309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7ed00-33a2-477f-a2ad-223924c3ce14 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:03.698449 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:03.743279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cf3d5-dba6-4b83-82d1-e36d4b6841e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:03.745118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cf3d5-dba6-4b83-82d1-e36d4b6841e6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:03.745118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cf3d5-dba6-4b83-82d1-e36d4b6841e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:03.745118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cf3d5-dba6-4b83-82d1-e36d4b6841e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:03.746695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cf3d5-dba6-4b83-82d1-e36d4b6841e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:03.747043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cf3d5-dba6-4b83-82d1-e36d4b6841e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:03.748450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cf3d5-dba6-4b83-82d1-e36d4b6841e6 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:03.759602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 219/440] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:45:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:03.760922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92778bff-c669-4c09-adad-36804ece7b58 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:03.761209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92778bff-c669-4c09-adad-36804ece7b58 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:03.761484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92778bff-c669-4c09-adad-36804ece7b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:03.779716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-92778bff-c669-4c09-adad-36804ece7b58 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:03.779900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 222/441] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:03.780519 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d81657-a44c-42e8-a414-620261b6af4c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:03.780826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-92778bff-c669-4c09-adad-36804ece7b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:03.795263 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:03.849291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 220/442] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:03.850387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92778bff-c669-4c09-adad-36804ece7b58 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:03.883329 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d81657-a44c-42e8-a414-620261b6af4c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:03.921491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd20af2-362a-44f2-ad10-9bc5a0e4db1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:03.922250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd20af2-362a-44f2-ad10-9bc5a0e4db1e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:03.922530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd20af2-362a-44f2-ad10-9bc5a0e4db1e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:03.922850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd20af2-362a-44f2-ad10-9bc5a0e4db1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:03.935060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:03.967919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92778bff-c669-4c09-adad-36804ece7b58 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:03.997669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd20af2-362a-44f2-ad10-9bc5a0e4db1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:03.999203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd20af2-362a-44f2-ad10-9bc5a0e4db1e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:04.007733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35642cb6-d92c-45fb-93c1-728e787f60b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:04.008620 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35642cb6-d92c-45fb-93c1-728e787f60b9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:04.008901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35642cb6-d92c-45fb-93c1-728e787f60b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:04.009176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35642cb6-d92c-45fb-93c1-728e787f60b9 None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:04.010417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35642cb6-d92c-45fb-93c1-728e787f60b9 None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:04.011891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35642cb6-d92c-45fb-93c1-728e787f60b9 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:04.014208 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 221/443] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:45:03 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:04.014375 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35642cb6-d92c-45fb-93c1-728e787f60b9 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:04.034911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:04.115788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 223/444] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:04.117144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd20af2-362a-44f2-ad10-9bc5a0e4db1e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:04.137192 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35642cb6-d92c-45fb-93c1-728e787f60b9 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:04.166101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:04.176095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08c0cf-6c6f-4ade-8a3c-03402b56ca36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:04.176912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08c0cf-6c6f-4ade-8a3c-03402b56ca36 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:04.177189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08c0cf-6c6f-4ade-8a3c-03402b56ca36 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:04.177458 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08c0cf-6c6f-4ade-8a3c-03402b56ca36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:04.188069 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08c0cf-6c6f-4ade-8a3c-03402b56ca36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-7d08c0cf-6c6f-4ade-8a3c-03402b56ca36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:04.195876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:04.196830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:04.197605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 222/445] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:04.198258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08c0cf-6c6f-4ade-8a3c-03402b56ca36 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:04.231637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd20af2-362a-44f2-ad10-9bc5a0e4db1e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:04.275107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041a947-b272-4c2e-997f-37c2e97ca857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:04.275985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a947-b272-4c2e-997f-37c2e97ca857 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:04.276255 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a947-b272-4c2e-997f-37c2e97ca857 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:04.276523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a947-b272-4c2e-997f-37c2e97ca857 None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:04.277758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041a947-b272-4c2e-997f-37c2e97ca857 None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:04.279270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041a947-b272-4c2e-997f-37c2e97ca857 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:04.280927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 224/446] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:45:04 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:04.281681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041a947-b272-4c2e-997f-37c2e97ca857 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:04.285470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:04.314811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08c0cf-6c6f-4ade-8a3c-03402b56ca36 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:04.353776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aae2e4-ec9a-4834-acca-98e9dd805dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:04.354703 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aae2e4-ec9a-4834-acca-98e9dd805dec None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:04.354987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aae2e4-ec9a-4834-acca-98e9dd805dec None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:04.355285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aae2e4-ec9a-4834-acca-98e9dd805dec None admin] PATH_INFO: `/v3/projects/ea40190bfcd344248c8e0d7f7860ef36` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:04.356512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aae2e4-ec9a-4834-acca-98e9dd805dec None admin] RBAC: Authorizing `identity:get_project(project_id=ea40190bfcd344248c8e0d7f7860ef36)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:04.357948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aae2e4-ec9a-4834-acca-98e9dd805dec None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:04.359696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 223/447] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:45:04 2022] GET /identity/v3/projects/ea40190bfcd344248c8e0d7f7860ef36 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:04.360459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aae2e4-ec9a-4834-acca-98e9dd805dec None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:04.391290 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041a947-b272-4c2e-997f-37c2e97ca857 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:04.401139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:04.426341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa06aa-b8e0-4baf-a5ff-5944452c5775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:04.427052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa06aa-b8e0-4baf-a5ff-5944452c5775 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:04.427342 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa06aa-b8e0-4baf-a5ff-5944452c5775 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:04.427612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa06aa-b8e0-4baf-a5ff-5944452c5775 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:04.428377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa06aa-b8e0-4baf-a5ff-5944452c5775 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:04.428683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa06aa-b8e0-4baf-a5ff-5944452c5775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:04.430068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa06aa-b8e0-4baf-a5ff-5944452c5775 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:04.438641 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 225/448] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:04.439497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa06aa-b8e0-4baf-a5ff-5944452c5775 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:04.462467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aae2e4-ec9a-4834-acca-98e9dd805dec None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:04.496801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f03d1b-0509-4ec8-a120-aaa8635017a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:04.497470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f03d1b-0509-4ec8-a120-aaa8635017a1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:04.497753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f03d1b-0509-4ec8-a120-aaa8635017a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:04.498031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f03d1b-0509-4ec8-a120-aaa8635017a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:04.498896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f03d1b-0509-4ec8-a120-aaa8635017a1 None admin] RBAC: Adding query filter params (group.id=8327f367eb2848039fa0c9e8eae691d0, role.id=f819d0f29adf41a49fa045dc3c031527, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:04.499212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f03d1b-0509-4ec8-a120-aaa8635017a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:04.500666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f03d1b-0509-4ec8-a120-aaa8635017a1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:04.506762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 224/449] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 09:45:04 2022] GET /identity/v3/role_assignments?group.id=8327f367eb2848039fa0c9e8eae691d0&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=f819d0f29adf41a49fa045dc3c031527 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:04.507569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f03d1b-0509-4ec8-a120-aaa8635017a1 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:04.520414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:04.550113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa06aa-b8e0-4baf-a5ff-5944452c5775 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:04.588467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349b1ae-878b-4fb2-98c4-15fa01fc6228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:04.589355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349b1ae-878b-4fb2-98c4-15fa01fc6228 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:04.589635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349b1ae-878b-4fb2-98c4-15fa01fc6228 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:04.589909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349b1ae-878b-4fb2-98c4-15fa01fc6228 None admin] PATH_INFO: `/v3/groups/8327f367eb2848039fa0c9e8eae691d0` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:04.591235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349b1ae-878b-4fb2-98c4-15fa01fc6228 None admin] RBAC: Authorizing `identity:get_group(group_id=8327f367eb2848039fa0c9e8eae691d0)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:04.592757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349b1ae-878b-4fb2-98c4-15fa01fc6228 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:04.594602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 226/450] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:45:04 2022] GET /identity/v3/groups/8327f367eb2848039fa0c9e8eae691d0 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:04.601025 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349b1ae-878b-4fb2-98c4-15fa01fc6228 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:04.622930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f03d1b-0509-4ec8-a120-aaa8635017a1 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:04.647158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:04.665750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2c2d3-f30a-4bec-a0ce-16b669f61813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:04.666419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2c2d3-f30a-4bec-a0ce-16b669f61813 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:04.666733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2c2d3-f30a-4bec-a0ce-16b669f61813 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:04.667004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2c2d3-f30a-4bec-a0ce-16b669f61813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:04.722843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349b1ae-878b-4fb2-98c4-15fa01fc6228 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:04.744720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2c2d3-f30a-4bec-a0ce-16b669f61813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:04.746193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2c2d3-f30a-4bec-a0ce-16b669f61813 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:04.756450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:04.759388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62408-6722-4c64-80c9-54f3bf6974b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:04.760049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62408-6722-4c64-80c9-54f3bf6974b4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:04.760331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62408-6722-4c64-80c9-54f3bf6974b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:04.760613 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62408-6722-4c64-80c9-54f3bf6974b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:04.761451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62408-6722-4c64-80c9-54f3bf6974b4 None admin] RBAC: Adding query filter params (group.id=8327f367eb2848039fa0c9e8eae691d0, role.id=f7f18826d4b44c1284ac31e25088afbd, scope.project.id=ea40190bfcd344248c8e0d7f7860ef36) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:04.761764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62408-6722-4c64-80c9-54f3bf6974b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:04.763248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62408-6722-4c64-80c9-54f3bf6974b4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:04.769356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 227/451] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 09:45:04 2022] GET /identity/v3/role_assignments?group.id=8327f367eb2848039fa0c9e8eae691d0&scope.project.id=ea40190bfcd344248c8e0d7f7860ef36&role.id=f7f18826d4b44c1284ac31e25088afbd => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:04.855034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 225/452] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:04.860818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62408-6722-4c64-80c9-54f3bf6974b4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:04.967146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62408-6722-4c64-80c9-54f3bf6974b4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:04.968826 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:05.003191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e154619-d471-48c6-8441-2649336b0629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:05.004083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e154619-d471-48c6-8441-2649336b0629 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:05.004354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e154619-d471-48c6-8441-2649336b0629 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:05.004622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e154619-d471-48c6-8441-2649336b0629 None admin] PATH_INFO: `/v3/projects/d79cb8b2f46e418d955fc3be77ed8c7a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:05.005857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e154619-d471-48c6-8441-2649336b0629 None admin] RBAC: Authorizing `identity:get_project(project_id=d79cb8b2f46e418d955fc3be77ed8c7a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:05.007350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e154619-d471-48c6-8441-2649336b0629 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:05.009043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 228/453] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:45:04 2022] GET /identity/v3/projects/d79cb8b2f46e418d955fc3be77ed8c7a => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:05.014988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2c2d3-f30a-4bec-a0ce-16b669f61813 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:05.131415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2c2d3-f30a-4bec-a0ce-16b669f61813 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:05.141287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:05.168504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b2989-01e6-410e-905d-ebb6abac2ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:05.169368 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b2989-01e6-410e-905d-ebb6abac2ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:05.169645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b2989-01e6-410e-905d-ebb6abac2ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:05.169909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b2989-01e6-410e-905d-ebb6abac2ae1 None admin] PATH_INFO: `/v3/groups/ca38cf14c3b2434ab7b496ada0049811` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:05.171098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b2989-01e6-410e-905d-ebb6abac2ae1 None admin] RBAC: Authorizing `identity:get_group(group_id=ca38cf14c3b2434ab7b496ada0049811)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:05.172536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b2989-01e6-410e-905d-ebb6abac2ae1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:05.175039 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 226/454] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 09:45:05 2022] GET /identity/v3/groups/ca38cf14c3b2434ab7b496ada0049811 => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:05.181233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e154619-d471-48c6-8441-2649336b0629 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:05.289520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e154619-d471-48c6-8441-2649336b0629 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:05.322822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45782c-c901-4068-9e14-1accc5c3f3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:05.323477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45782c-c901-4068-9e14-1accc5c3f3ec None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:05.323758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45782c-c901-4068-9e14-1accc5c3f3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:05.324028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45782c-c901-4068-9e14-1accc5c3f3ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:05.324861 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45782c-c901-4068-9e14-1accc5c3f3ec None admin] RBAC: Adding query filter params (group.id=ca38cf14c3b2434ab7b496ada0049811, role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=d79cb8b2f46e418d955fc3be77ed8c7a) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:05.325172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45782c-c901-4068-9e14-1accc5c3f3ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:05.326637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45782c-c901-4068-9e14-1accc5c3f3ec None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:05.332605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 229/455] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 09:45:05 2022] GET /identity/v3/role_assignments?group.id=ca38cf14c3b2434ab7b496ada0049811&scope.project.id=d79cb8b2f46e418d955fc3be77ed8c7a&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:07.728410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2183a5-7cc6-43b5-a390-c6db443411f4 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:07.728938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2183a5-7cc6-43b5-a390-c6db443411f4 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:07.728938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2183a5-7cc6-43b5-a390-c6db443411f4 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:07.730625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 227/456] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:07.744298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0edb-df2e-479f-9278-e8bd0f4e5665 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:07.745218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0edb-df2e-479f-9278-e8bd0f4e5665 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:07.745700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0edb-df2e-479f-9278-e8bd0f4e5665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:07.771630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-25dc0edb-df2e-479f-9278-e8bd0f4e5665 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:07.773222 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-25dc0edb-df2e-479f-9278-e8bd0f4e5665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:07.813514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:07.863849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202464a8-903d-4ed3-966b-e641707bc460 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:07.864600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202464a8-903d-4ed3-966b-e641707bc460 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:07.865105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202464a8-903d-4ed3-966b-e641707bc460 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:07.866903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 228/457] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:07.870865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 230/458] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:07.874538 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957655-300b-4c6c-a24a-dd94b110767d None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:07.875189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957655-300b-4c6c-a24a-dd94b110767d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:07.875809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957655-300b-4c6c-a24a-dd94b110767d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:07.896734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc0edb-df2e-479f-9278-e8bd0f4e5665 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:07.923510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-73957655-300b-4c6c-a24a-dd94b110767d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:07.924764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-73957655-300b-4c6c-a24a-dd94b110767d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:08.038245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc0edb-df2e-479f-9278-e8bd0f4e5665 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:08.093521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917a4fc-3e37-4eec-aaa0-d4d2eb1364d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:08.095005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917a4fc-3e37-4eec-aaa0-d4d2eb1364d0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:08.095353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917a4fc-3e37-4eec-aaa0-d4d2eb1364d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:08.095636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917a4fc-3e37-4eec-aaa0-d4d2eb1364d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:08.107522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917a4fc-3e37-4eec-aaa0-d4d2eb1364d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:08.109307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917a4fc-3e37-4eec-aaa0-d4d2eb1364d0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-1917a4fc-3e37-4eec-aaa0-d4d2eb1364d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:08.129781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:08.130737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:08.131568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:08.131568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:08.131568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:08.131568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:08.131568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 231/459] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:08.137196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917a4fc-3e37-4eec-aaa0-d4d2eb1364d0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:08.150617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 229/460] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:08.157300 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:08.204529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73957655-300b-4c6c-a24a-dd94b110767d None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:08.250700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917a4fc-3e37-4eec-aaa0-d4d2eb1364d0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:08.290696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8805f84-580c-4aa9-aecd-c4d9c1e396c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:08.291283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8805f84-580c-4aa9-aecd-c4d9c1e396c7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:08.291564 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8805f84-580c-4aa9-aecd-c4d9c1e396c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:08.291840 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8805f84-580c-4aa9-aecd-c4d9c1e396c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:08.292581 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8805f84-580c-4aa9-aecd-c4d9c1e396c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:08.292886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8805f84-580c-4aa9-aecd-c4d9c1e396c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:08.294300 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8805f84-580c-4aa9-aecd-c4d9c1e396c7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:08.336199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 232/461] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:08.343431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8805f84-580c-4aa9-aecd-c4d9c1e396c7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:08.424575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73957655-300b-4c6c-a24a-dd94b110767d None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:08.447066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:08.447775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:08.477264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8805f84-580c-4aa9-aecd-c4d9c1e396c7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:08.500116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25949565-b222-4cda-aa6b-14d8e04a44c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:08.500942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25949565-b222-4cda-aa6b-14d8e04a44c8 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:08.501220 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25949565-b222-4cda-aa6b-14d8e04a44c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:08.501572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25949565-b222-4cda-aa6b-14d8e04a44c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:08.522450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25949565-b222-4cda-aa6b-14d8e04a44c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:08.523989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25949565-b222-4cda-aa6b-14d8e04a44c8 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:08.547505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257112bd-b7dd-4dd4-bc3b-367bc073d54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:08.548112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257112bd-b7dd-4dd4-bc3b-367bc073d54a None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:08.548419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257112bd-b7dd-4dd4-bc3b-367bc073d54a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:08.548702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257112bd-b7dd-4dd4-bc3b-367bc073d54a None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:08.549440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257112bd-b7dd-4dd4-bc3b-367bc073d54a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:08.550956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257112bd-b7dd-4dd4-bc3b-367bc073d54a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-25949565-b222-4cda-aa6b-14d8e04a44c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:08.556808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:08.557634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:08.558475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:08.558475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:08.558475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:08.558475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:08.564115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 230/462] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:08.564115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709f918-d449-4416-852e-6f29c87ac658 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:08.564115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709f918-d449-4416-852e-6f29c87ac658 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:08.564115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709f918-d449-4416-852e-6f29c87ac658 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:08.564115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 231/463] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:08.566143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709f918-d449-4416-852e-6f29c87ac658 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:08.583857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 233/464] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:45:08 2022] POST /identity/v3/users => generated 246 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:45:08.585553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330e491-b8aa-45fc-b74f-431bbd9fa209 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:08.585844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330e491-b8aa-45fc-b74f-431bbd9fa209 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:08.586123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330e491-b8aa-45fc-b74f-431bbd9fa209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:08.609979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1330e491-b8aa-45fc-b74f-431bbd9fa209 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:08.611685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1330e491-b8aa-45fc-b74f-431bbd9fa209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:08.712782 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709f918-d449-4416-852e-6f29c87ac658 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:08.731146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 234/465] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:08.733153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac000d6f-2385-4a57-809d-4ea12124b302 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:08.733653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac000d6f-2385-4a57-809d-4ea12124b302 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:08.734140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac000d6f-2385-4a57-809d-4ea12124b302 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:08.735946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 235/466] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:08.741901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0b32-3ebb-4921-baba-31683a8d50c7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:08.742518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0b32-3ebb-4921-baba-31683a8d50c7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:08.743013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0b32-3ebb-4921-baba-31683a8d50c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:08.767669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ae3d0b32-3ebb-4921-baba-31683a8d50c7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:08.769234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cc460-eed1-4c49-9d93-00db6f49c2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:08.769998 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cc460-eed1-4c49-9d93-00db6f49c2fc None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:08.770282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cc460-eed1-4c49-9d93-00db6f49c2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:08.770381 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ae3d0b32-3ebb-4921-baba-31683a8d50c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:08.770631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cc460-eed1-4c49-9d93-00db6f49c2fc None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:08.771389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cc460-eed1-4c49-9d93-00db6f49c2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:08.771715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cc460-eed1-4c49-9d93-00db6f49c2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:08.774330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cc460-eed1-4c49-9d93-00db6f49c2fc None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:08.789099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 232/467] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:08.789802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cc460-eed1-4c49-9d93-00db6f49c2fc None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:08.812034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:08.843887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:08.861211 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 236/468] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:08.862185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d0b32-3ebb-4921-baba-31683a8d50c7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:08.935428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cc460-eed1-4c49-9d93-00db6f49c2fc None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:08.984874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d6322-afc8-4dac-ae59-c7323f600dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:08.985160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d0b32-3ebb-4921-baba-31683a8d50c7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:08.985692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d6322-afc8-4dac-ae59-c7323f600dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:08.986123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d6322-afc8-4dac-ae59-c7323f600dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:08.986502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d6322-afc8-4dac-ae59-c7323f600dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:09.000840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d6322-afc8-4dac-ae59-c7323f600dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:09.003123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d6322-afc8-4dac-ae59-c7323f600dd2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:09.023206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369dff0-e73f-430b-8992-87c181b29c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:09.023992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369dff0-e73f-430b-8992-87c181b29c26 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:09.024345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369dff0-e73f-430b-8992-87c181b29c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:09.024694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369dff0-e73f-430b-8992-87c181b29c26 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-916d6322-afc8-4dac-ae59-c7323f600dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:09.025401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:09.026219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:09.027080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:09.027080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:09.027080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:09.027080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:09.027284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369dff0-e73f-430b-8992-87c181b29c26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:09.027350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 233/469] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:09.027413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369dff0-e73f-430b-8992-87c181b29c26 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:09.027773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d6322-afc8-4dac-ae59-c7323f600dd2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:09.069683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 237/470] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:45:08 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:45:09.071130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369dff0-e73f-430b-8992-87c181b29c26 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:09.119733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:09.145758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:09.164976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d6322-afc8-4dac-ae59-c7323f600dd2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:09.207457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369dff0-e73f-430b-8992-87c181b29c26 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:09.217745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22d221-387b-4bfe-9707-e439094e7d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:09.219366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22d221-387b-4bfe-9707-e439094e7d68 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:09.220459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22d221-387b-4bfe-9707-e439094e7d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:09.220959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22d221-387b-4bfe-9707-e439094e7d68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:09.240367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22d221-387b-4bfe-9707-e439094e7d68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:09.243041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22d221-387b-4bfe-9707-e439094e7d68 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:09.249952 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81c759-340c-4dbf-9a68-5d55527533fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:09.250835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81c759-340c-4dbf-9a68-5d55527533fc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:09.251186 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81c759-340c-4dbf-9a68-5d55527533fc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:09.251667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81c759-340c-4dbf-9a68-5d55527533fc None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:09.252503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81c759-340c-4dbf-9a68-5d55527533fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:09.252880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81c759-340c-4dbf-9a68-5d55527533fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:09.254552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81c759-340c-4dbf-9a68-5d55527533fc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-ac22d221-387b-4bfe-9707-e439094e7d68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:09.268926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:09.270022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:09.271029 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:09.271029 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:09.271029 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:09.271029 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:09.271239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 238/471] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:09.273057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 234/472] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:09.276150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53b603-e8cc-44cc-85aa-068e02e2abce None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:09.276440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53b603-e8cc-44cc-85aa-068e02e2abce None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:09.276717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53b603-e8cc-44cc-85aa-068e02e2abce None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:09.278806 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22d221-387b-4bfe-9707-e439094e7d68 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:09.279807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 239/473] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:09.280826 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53b603-e8cc-44cc-85aa-068e02e2abce None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:09.421500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22d221-387b-4bfe-9707-e439094e7d68 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:09.443892 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53b603-e8cc-44cc-85aa-068e02e2abce None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:09.449741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:09.462031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b8f32-83a9-4e47-8f89-02c8da3a9cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:09.462840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b8f32-83a9-4e47-8f89-02c8da3a9cd1 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:09.463191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b8f32-83a9-4e47-8f89-02c8da3a9cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:09.463534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b8f32-83a9-4e47-8f89-02c8da3a9cd1 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:09.464626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:09.465608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b8f32-83a9-4e47-8f89-02c8da3a9cd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:09.467706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b8f32-83a9-4e47-8f89-02c8da3a9cd1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:09.496770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 235/474] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:45:09 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:45:09.498919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff49cd1-6555-4773-afda-b7e55a97525e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:09.499309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff49cd1-6555-4773-afda-b7e55a97525e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:09.499902 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff49cd1-6555-4773-afda-b7e55a97525e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:09.504793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d941f-5bba-4dfd-8db0-e770f7007875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:09.505769 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d941f-5bba-4dfd-8db0-e770f7007875 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:09.506282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d941f-5bba-4dfd-8db0-e770f7007875 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:09.506881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d941f-5bba-4dfd-8db0-e770f7007875 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:09.507850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d941f-5bba-4dfd-8db0-e770f7007875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:09.508369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d941f-5bba-4dfd-8db0-e770f7007875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:09.509987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d941f-5bba-4dfd-8db0-e770f7007875 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:09.520493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-5ff49cd1-6555-4773-afda-b7e55a97525e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:09.522430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5ff49cd1-6555-4773-afda-b7e55a97525e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:09.525278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 240/475] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:09.530629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d941f-5bba-4dfd-8db0-e770f7007875 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:09.657984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 236/476] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:09.702469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d941f-5bba-4dfd-8db0-e770f7007875 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:09.712009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff49cd1-6555-4773-afda-b7e55a97525e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:09.757436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb7184-4f9b-46fd-b1b4-4b49d103ff9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:09.758495 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb7184-4f9b-46fd-b1b4-4b49d103ff9e None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:09.759039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb7184-4f9b-46fd-b1b4-4b49d103ff9e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:09.759498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb7184-4f9b-46fd-b1b4-4b49d103ff9e None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:09.760474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb7184-4f9b-46fd-b1b4-4b49d103ff9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:09.762153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb7184-4f9b-46fd-b1b4-4b49d103ff9e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:09.787996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 241/477] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:45:09 2022] POST /identity/v3/users => generated 249 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:45:09.850772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff49cd1-6555-4773-afda-b7e55a97525e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:09.864305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:09.895911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab397c8-a519-435d-ad63-49ea659c8778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:09.897033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab397c8-a519-435d-ad63-49ea659c8778 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:09.897533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab397c8-a519-435d-ad63-49ea659c8778 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:09.897991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab397c8-a519-435d-ad63-49ea659c8778 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:09.899047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab397c8-a519-435d-ad63-49ea659c8778 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:09.900705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab397c8-a519-435d-ad63-49ea659c8778 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:09.909411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.sql.core [None req-aab397c8-a519-435d-ad63-49ea659c8778 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 09:45:09.909411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 29 09:45:09.909411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [parameters: {'id': 'f566723b5fdc4637a975896d75df0d23', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 29 09:45:09.909411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80176) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-aab397c8-a519-435d-ad63-49ea659c8778 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application conn.query(q) Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application result.read() Jun 29 09:45:09.928107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return ref.to_dict() Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 09:45:09.929028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application next(self.gen) Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application yield resource Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application next(self.gen) Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise self.value Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application session.commit() Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 09:45:09.930013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self.session.flush() Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application compat.raise_( Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 09:45:09.930925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 09:45:09.932560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application conn.query(q) Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application result.read() Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 09:45:09.933341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [parameters: {'id': 'f566723b5fdc4637a975896d75df0d23', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:09.934115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 29 09:45:09.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:09.940843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 237/478] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jun 29 09:45:09 2022] POST /identity/v3/roles => generated 144 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:09.946894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb7184-4f9b-46fd-b1b4-4b49d103ff9e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:09.983600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:10.076318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb7184-4f9b-46fd-b1b4-4b49d103ff9e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:10.113916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37060694-bbec-4315-baca-b614454100db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:10.114774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37060694-bbec-4315-baca-b614454100db None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:10.115051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37060694-bbec-4315-baca-b614454100db None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:10.115322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37060694-bbec-4315-baca-b614454100db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:10.125959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37060694-bbec-4315-baca-b614454100db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-37060694-bbec-4315-baca-b614454100db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:10.134491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:10.135456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:10.136235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 242/479] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:10.142924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab397c8-a519-435d-ad63-49ea659c8778 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:10.249697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab397c8-a519-435d-ad63-49ea659c8778 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:10.285264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fefb13-52b1-4866-b923-10603b125a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:10.285945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fefb13-52b1-4866-b923-10603b125a21 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:10.286223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fefb13-52b1-4866-b923-10603b125a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:10.286495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fefb13-52b1-4866-b923-10603b125a21 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:10.287273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fefb13-52b1-4866-b923-10603b125a21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:10.287576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fefb13-52b1-4866-b923-10603b125a21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:10.288946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fefb13-52b1-4866-b923-10603b125a21 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:10.297861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 238/480] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:11.028350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e76837-7660-4fe0-b00f-8535f710862b None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:11.029059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e76837-7660-4fe0-b00f-8535f710862b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:11.029059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e76837-7660-4fe0-b00f-8535f710862b None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:11.030498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 243/481] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:11.036407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680f36f-97cb-4b7b-9cff-bed476283bb7 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:11.036973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680f36f-97cb-4b7b-9cff-bed476283bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:11.036973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680f36f-97cb-4b7b-9cff-bed476283bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:11.063813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-3680f36f-97cb-4b7b-9cff-bed476283bb7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:11.065196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3680f36f-97cb-4b7b-9cff-bed476283bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:11.108748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:11.173739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 239/482] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:11.207423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e76837-7660-4fe0-b00f-8535f710862b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:11.331355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:11.359683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e76837-7660-4fe0-b00f-8535f710862b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:11.409347 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e35cc-53b8-4bcd-8ff7-34b58d00a14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:11.410172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e35cc-53b8-4bcd-8ff7-34b58d00a14c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:11.410460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e35cc-53b8-4bcd-8ff7-34b58d00a14c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:11.410766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e35cc-53b8-4bcd-8ff7-34b58d00a14c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:11.431525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e35cc-53b8-4bcd-8ff7-34b58d00a14c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-233e35cc-53b8-4bcd-8ff7-34b58d00a14c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:11.439423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:11.440311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:11.441563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 244/483] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:11.446799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680f36f-97cb-4b7b-9cff-bed476283bb7 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:11.605560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680f36f-97cb-4b7b-9cff-bed476283bb7 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:11.665292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254eb762-abb0-43f5-906a-0d83d07f26d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:11.665980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb762-abb0-43f5-906a-0d83d07f26d6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:11.666263 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb762-abb0-43f5-906a-0d83d07f26d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:11.666540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb762-abb0-43f5-906a-0d83d07f26d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:11.667323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254eb762-abb0-43f5-906a-0d83d07f26d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:11.667631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254eb762-abb0-43f5-906a-0d83d07f26d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:11.669012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254eb762-abb0-43f5-906a-0d83d07f26d6 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:11.685111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 240/484] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:11.690992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e35cc-53b8-4bcd-8ff7-34b58d00a14c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:11.741468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99cf0f-a0cc-4444-b077-c381c8d1ad20 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:11.741816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99cf0f-a0cc-4444-b077-c381c8d1ad20 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:11.742025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99cf0f-a0cc-4444-b077-c381c8d1ad20 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:11.743447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 241/485] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:11.753452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c53be-39d8-4d6c-8717-4abae52cbd2d None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:11.753733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c53be-39d8-4d6c-8717-4abae52cbd2d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:11.754011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c53be-39d8-4d6c-8717-4abae52cbd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:11.782476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a04c53be-39d8-4d6c-8717-4abae52cbd2d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:11.783798 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a04c53be-39d8-4d6c-8717-4abae52cbd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:11.830945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:11.855254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e35cc-53b8-4bcd-8ff7-34b58d00a14c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:11.906064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdb6e6-03bf-4556-aad7-f514da7ac374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:11.906835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdb6e6-03bf-4556-aad7-f514da7ac374 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:11.907119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdb6e6-03bf-4556-aad7-f514da7ac374 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:11.907615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdb6e6-03bf-4556-aad7-f514da7ac374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:11.923016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 242/486] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:11.928563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdb6e6-03bf-4556-aad7-f514da7ac374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:11.930100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdb6e6-03bf-4556-aad7-f514da7ac374 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:11.938997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-2fbdb6e6-03bf-4556-aad7-f514da7ac374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:11.949544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:11.950403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:11.951464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:11.951464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:11.951464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:11.951464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:11.951464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 245/487] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:11.957485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c53be-39d8-4d6c-8717-4abae52cbd2d None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:11.959658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdb6e6-03bf-4556-aad7-f514da7ac374 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:12.103232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c53be-39d8-4d6c-8717-4abae52cbd2d None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:12.151651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2833d4f-c924-4c70-b90c-93a27493d30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:12.152337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2833d4f-c924-4c70-b90c-93a27493d30a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.152613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2833d4f-c924-4c70-b90c-93a27493d30a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.152885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2833d4f-c924-4c70-b90c-93a27493d30a None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.153624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2833d4f-c924-4c70-b90c-93a27493d30a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:12.153926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2833d4f-c924-4c70-b90c-93a27493d30a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:12.155327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2833d4f-c924-4c70-b90c-93a27493d30a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:12.178366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 243/488] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:12.184441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2833d4f-c924-4c70-b90c-93a27493d30a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:12.205712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdb6e6-03bf-4556-aad7-f514da7ac374 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:12.216905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:12.265195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0c2e7-4b78-46cc-9fca-d4d0df6cd612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:12.266177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0c2e7-4b78-46cc-9fca-d4d0df6cd612 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.266551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0c2e7-4b78-46cc-9fca-d4d0df6cd612 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.266940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0c2e7-4b78-46cc-9fca-d4d0df6cd612 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.281663 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0c2e7-4b78-46cc-9fca-d4d0df6cd612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-05c0c2e7-4b78-46cc-9fca-d4d0df6cd612 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:12.289791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:12.290733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:12.291538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 246/489] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:12.298244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a685920-a0ea-43bb-8192-63ee2cc6f4c8 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.298244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a685920-a0ea-43bb-8192-63ee2cc6f4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.298244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a685920-a0ea-43bb-8192-63ee2cc6f4c8 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.298244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 247/490] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:12.298244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a685920-a0ea-43bb-8192-63ee2cc6f4c8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:12.340057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2833d4f-c924-4c70-b90c-93a27493d30a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:12.360236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:12.389553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d551fa8-d2eb-4e9f-b792-463ff7ef64d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:12.391280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d551fa8-d2eb-4e9f-b792-463ff7ef64d3 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.391705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d551fa8-d2eb-4e9f-b792-463ff7ef64d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.392153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d551fa8-d2eb-4e9f-b792-463ff7ef64d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.403292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d551fa8-d2eb-4e9f-b792-463ff7ef64d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:12.405453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d551fa8-d2eb-4e9f-b792-463ff7ef64d3 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:12.418598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a685920-a0ea-43bb-8192-63ee2cc6f4c8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-1d551fa8-d2eb-4e9f-b792-463ff7ef64d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:12.425986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:12.427148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:12.428238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:12.428238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:12.428238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:12.428238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:12.428238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:12.428238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:12.428238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:12.428238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:12.428238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:12.428238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:12.434380 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 244/491] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 09:45:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:12.435939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f41d9-399a-4ab9-9444-53f41271b5bf None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.436276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f41d9-399a-4ab9-9444-53f41271b5bf None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.436688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f41d9-399a-4ab9-9444-53f41271b5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.454162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-851f41d9-399a-4ab9-9444-53f41271b5bf None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:12.455674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-851f41d9-399a-4ab9-9444-53f41271b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:12.462508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9dea17-8184-41c9-ae5e-32c2d0e1e4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:12.463461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dea17-8184-41c9-ae5e-32c2d0e1e4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.464442 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dea17-8184-41c9-ae5e-32c2d0e1e4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.464891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dea17-8184-41c9-ae5e-32c2d0e1e4f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.465823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9dea17-8184-41c9-ae5e-32c2d0e1e4f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:12.466763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9dea17-8184-41c9-ae5e-32c2d0e1e4f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:12.468826 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9dea17-8184-41c9-ae5e-32c2d0e1e4f3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:12.481133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 248/492] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:12.482208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9dea17-8184-41c9-ae5e-32c2d0e1e4f3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:12.542014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 245/493] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:12.543136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f41d9-399a-4ab9-9444-53f41271b5bf None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:12.561140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:12.589170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9dea17-8184-41c9-ae5e-32c2d0e1e4f3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:12.608752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:12.628523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5e0d8-abd2-40ae-8ff0-ecf176ea04c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:12.629470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5e0d8-abd2-40ae-8ff0-ecf176ea04c1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.629923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5e0d8-abd2-40ae-8ff0-ecf176ea04c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.630337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5e0d8-abd2-40ae-8ff0-ecf176ea04c1 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.631341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5e0d8-abd2-40ae-8ff0-ecf176ea04c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:12.631798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5e0d8-abd2-40ae-8ff0-ecf176ea04c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:12.633384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5e0d8-abd2-40ae-8ff0-ecf176ea04c1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:12.661388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f41d9-399a-4ab9-9444-53f41271b5bf None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:12.672770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 249/494] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 09:45:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:12.674293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c82e7-63e0-40cd-9024-ec5af323103e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.674689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c82e7-63e0-40cd-9024-ec5af323103e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.674965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c82e7-63e0-40cd-9024-ec5af323103e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.676410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 250/495] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:12.679695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c82e7-63e0-40cd-9024-ec5af323103e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:12.702665 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f229ae1-1a18-4b37-acc0-ee41076d510e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:12.703420 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229ae1-1a18-4b37-acc0-ee41076d510e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.703699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229ae1-1a18-4b37-acc0-ee41076d510e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.703980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229ae1-1a18-4b37-acc0-ee41076d510e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.715323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f229ae1-1a18-4b37-acc0-ee41076d510e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:12.717179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f229ae1-1a18-4b37-acc0-ee41076d510e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-8f229ae1-1a18-4b37-acc0-ee41076d510e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:12.737647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:12.738636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:12.739475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:12.739475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:12.739475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:12.739475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:12.739475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 246/496] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:12.739745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f229ae1-1a18-4b37-acc0-ee41076d510e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:12.799481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:12.804548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c82e7-63e0-40cd-9024-ec5af323103e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:12.841984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e650dcc-42e4-4c52-9db3-db0668d69704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:12.842830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e650dcc-42e4-4c52-9db3-db0668d69704 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.843106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e650dcc-42e4-4c52-9db3-db0668d69704 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.843370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e650dcc-42e4-4c52-9db3-db0668d69704 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.846239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f229ae1-1a18-4b37-acc0-ee41076d510e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:12.854814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e650dcc-42e4-4c52-9db3-db0668d69704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-8e650dcc-42e4-4c52-9db3-db0668d69704 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:12.862538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:12.863469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:12.864287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 251/497] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:12.865526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd83fb-872c-4874-bc09-b1b8b3958f55 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.865526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd83fb-872c-4874-bc09-b1b8b3958f55 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.865526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd83fb-872c-4874-bc09-b1b8b3958f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.883693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-85dd83fb-872c-4874-bc09-b1b8b3958f55 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:12.884878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-85dd83fb-872c-4874-bc09-b1b8b3958f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:12.889323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41eb46b-55c4-496c-9c91-65b67141cfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:12.889983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41eb46b-55c4-496c-9c91-65b67141cfb0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:12.890261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41eb46b-55c4-496c-9c91-65b67141cfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:12.890533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41eb46b-55c4-496c-9c91-65b67141cfb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:12.919144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:12.970677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 252/498] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:12.970677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd83fb-872c-4874-bc09-b1b8b3958f55 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:12.982337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41eb46b-55c4-496c-9c91-65b67141cfb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:12.983930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41eb46b-55c4-496c-9c91-65b67141cfb0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:13.049370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:13.074226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd83fb-872c-4874-bc09-b1b8b3958f55 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:13.099809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 247/499] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:13.105099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41eb46b-55c4-496c-9c91-65b67141cfb0 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:13.110180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fbea2-64b5-46c5-a8d2-ef29498a1e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:13.111015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fbea2-64b5-46c5-a8d2-ef29498a1e3a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.111457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fbea2-64b5-46c5-a8d2-ef29498a1e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.111738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fbea2-64b5-46c5-a8d2-ef29498a1e3a None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.112475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fbea2-64b5-46c5-a8d2-ef29498a1e3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:13.112782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fbea2-64b5-46c5-a8d2-ef29498a1e3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:13.114193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fbea2-64b5-46c5-a8d2-ef29498a1e3a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:13.130451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 253/500] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:13.130451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fbea2-64b5-46c5-a8d2-ef29498a1e3a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:13.157605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:13.212031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41eb46b-55c4-496c-9c91-65b67141cfb0 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:13.230898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fbea2-64b5-46c5-a8d2-ef29498a1e3a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:13.246486 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f876-5c3f-4d8c-853a-742b4bc11a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:13.247184 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f876-5c3f-4d8c-853a-742b4bc11a08 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.247455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f876-5c3f-4d8c-853a-742b4bc11a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.247724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f876-5c3f-4d8c-853a-742b4bc11a08 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.248495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568f876-5c3f-4d8c-853a-742b4bc11a08 None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:13.248791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568f876-5c3f-4d8c-853a-742b4bc11a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:13.250157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568f876-5c3f-4d8c-853a-742b4bc11a08 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:13.258432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 248/501] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:13.259266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f876-5c3f-4d8c-853a-742b4bc11a08 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:13.271218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f431418-421f-4454-99d0-5de6dbccf525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:13.272042 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f431418-421f-4454-99d0-5de6dbccf525 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.272325 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f431418-421f-4454-99d0-5de6dbccf525 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.272602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f431418-421f-4454-99d0-5de6dbccf525 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.280658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:13.284233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f431418-421f-4454-99d0-5de6dbccf525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-0f431418-421f-4454-99d0-5de6dbccf525 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:13.292000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:13.293104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:13.293903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 254/502] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:13.295068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f431418-421f-4454-99d0-5de6dbccf525 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:13.368762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f876-5c3f-4d8c-853a-742b4bc11a08 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:13.386035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:13.402203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f431418-421f-4454-99d0-5de6dbccf525 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:13.402912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ba2fc-33a6-4096-b960-35e4daf34d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:13.403753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ba2fc-33a6-4096-b960-35e4daf34d3a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.404030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ba2fc-33a6-4096-b960-35e4daf34d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.404317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ba2fc-33a6-4096-b960-35e4daf34d3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.415113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ba2fc-33a6-4096-b960-35e4daf34d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:13.416586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ba2fc-33a6-4096-b960-35e4daf34d3a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-e84ba2fc-33a6-4096-b960-35e4daf34d3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:13.434469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:13.435518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:13.436517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:13.436517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:13.436517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:13.436517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:13.436517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 249/503] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:13.436991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a3eaa-703d-4124-aef7-1381998437e8 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.437269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a3eaa-703d-4124-aef7-1381998437e8 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.437539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a3eaa-703d-4124-aef7-1381998437e8 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.442626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 250/504] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:13.442626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a3eaa-703d-4124-aef7-1381998437e8 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:13.445457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38dc318-337f-4d21-8e80-bb22fba77e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:13.446254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38dc318-337f-4d21-8e80-bb22fba77e1b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.446576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38dc318-337f-4d21-8e80-bb22fba77e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.446857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38dc318-337f-4d21-8e80-bb22fba77e1b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.455323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38dc318-337f-4d21-8e80-bb22fba77e1b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:13.457035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38dc318-337f-4d21-8e80-bb22fba77e1b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f38dc318-337f-4d21-8e80-bb22fba77e1b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:13.473372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:13.474901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 255/505] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:13.476019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38dc318-337f-4d21-8e80-bb22fba77e1b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:13.522308 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:13.556681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a3eaa-703d-4124-aef7-1381998437e8 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:13.579815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38dc318-337f-4d21-8e80-bb22fba77e1b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:13.591763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe61f4d-1531-4980-aaf4-f44a49435925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:13.592569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe61f4d-1531-4980-aaf4-f44a49435925 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.592846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe61f4d-1531-4980-aaf4-f44a49435925 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.593115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe61f4d-1531-4980-aaf4-f44a49435925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.604593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe61f4d-1531-4980-aaf4-f44a49435925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:13.606077 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe61f4d-1531-4980-aaf4-f44a49435925 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:13.614538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211c812-0f0f-4c1b-aa78-af9cbab972b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:13.615240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211c812-0f0f-4c1b-aa78-af9cbab972b7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.615524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211c812-0f0f-4c1b-aa78-af9cbab972b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.615798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211c812-0f0f-4c1b-aa78-af9cbab972b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.616549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211c812-0f0f-4c1b-aa78-af9cbab972b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:13.616855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211c812-0f0f-4c1b-aa78-af9cbab972b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:13.618364 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211c812-0f0f-4c1b-aa78-af9cbab972b7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-9fe61f4d-1531-4980-aaf4-f44a49435925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:13.623002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:13.623994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:13.624882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:13.624882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:13.624882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:13.624882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:13.624882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 251/506] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:13.625134 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe61f4d-1531-4980-aaf4-f44a49435925 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:13.633843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 256/507] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:13.633843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e2049-5d01-4edd-953b-dbe77bd5c611 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.633843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e2049-5d01-4edd-953b-dbe77bd5c611 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.633843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e2049-5d01-4edd-953b-dbe77bd5c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.648387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7a8e2049-5d01-4edd-953b-dbe77bd5c611 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:13.649624 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7a8e2049-5d01-4edd-953b-dbe77bd5c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:13.651816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:13.718645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 257/508] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:13.719994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e2049-5d01-4edd-953b-dbe77bd5c611 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:13.741659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe61f4d-1531-4980-aaf4-f44a49435925 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:13.766145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:13.777412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ff186-9f66-48b5-b3bf-6e694ebe8466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:13.778008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff186-9f66-48b5-b3bf-6e694ebe8466 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.778281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff186-9f66-48b5-b3bf-6e694ebe8466 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.778548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff186-9f66-48b5-b3bf-6e694ebe8466 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.779392 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ff186-9f66-48b5-b3bf-6e694ebe8466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:13.779595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ff186-9f66-48b5-b3bf-6e694ebe8466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:13.780958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ff186-9f66-48b5-b3bf-6e694ebe8466 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:13.793387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 252/509] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:13.794197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ff186-9f66-48b5-b3bf-6e694ebe8466 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:13.820215 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e2049-5d01-4edd-953b-dbe77bd5c611 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:13.853878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61ad10-79b4-4140-8010-946c9750ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:13.854550 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61ad10-79b4-4140-8010-946c9750ef3f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.854878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61ad10-79b4-4140-8010-946c9750ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.855118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61ad10-79b4-4140-8010-946c9750ef3f None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.855956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61ad10-79b4-4140-8010-946c9750ef3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:13.856261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61ad10-79b4-4140-8010-946c9750ef3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:13.857730 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61ad10-79b4-4140-8010-946c9750ef3f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:13.865610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:13.876013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 258/510] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 09:45:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:13.877583 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61ad10-79b4-4140-8010-946c9750ef3f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:13.899776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ff186-9f66-48b5-b3bf-6e694ebe8466 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:13.934408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76caa1b1-f923-44a8-84e1-daf4ddc3d8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:13.935099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa1b1-f923-44a8-84e1-daf4ddc3d8ef None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:13.935376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa1b1-f923-44a8-84e1-daf4ddc3d8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:13.935645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa1b1-f923-44a8-84e1-daf4ddc3d8ef None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:13.936359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76caa1b1-f923-44a8-84e1-daf4ddc3d8ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:13.936664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76caa1b1-f923-44a8-84e1-daf4ddc3d8ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:13.938035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76caa1b1-f923-44a8-84e1-daf4ddc3d8ef None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:13.950441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 253/511] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:13.951285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76caa1b1-f923-44a8-84e1-daf4ddc3d8ef None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:13.981543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61ad10-79b4-4140-8010-946c9750ef3f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:13.996579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:14.018287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c46493-f864-4661-9037-12f7568243e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:14.019304 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c46493-f864-4661-9037-12f7568243e0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:14.019641 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c46493-f864-4661-9037-12f7568243e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:14.019983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c46493-f864-4661-9037-12f7568243e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:14.030592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c46493-f864-4661-9037-12f7568243e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:14.032083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c46493-f864-4661-9037-12f7568243e0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-d2c46493-f864-4661-9037-12f7568243e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:14.050950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:14.051864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:14.052691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:14.052691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:14.052691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:14.052691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:14.052691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 259/512] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:14.053580 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c46493-f864-4661-9037-12f7568243e0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:14.067812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76caa1b1-f923-44a8-84e1-daf4ddc3d8ef None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:14.102212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c6729-957d-46f6-9ceb-c47da8cfe79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:14.103054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c6729-957d-46f6-9ceb-c47da8cfe79a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:14.103339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c6729-957d-46f6-9ceb-c47da8cfe79a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:14.103622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c6729-957d-46f6-9ceb-c47da8cfe79a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:14.106002 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:14.114712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c6729-957d-46f6-9ceb-c47da8cfe79a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:14.116210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c6729-957d-46f6-9ceb-c47da8cfe79a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-d11c6729-957d-46f6-9ceb-c47da8cfe79a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:14.133854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:14.134929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:14.135810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:14.135810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:14.135810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:14.135810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:14.135810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 254/513] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:14.136065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c6729-957d-46f6-9ceb-c47da8cfe79a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:14.160836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c46493-f864-4661-9037-12f7568243e0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:14.195656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5a1ff-c7a1-4bbf-a309-8ae68c999c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:14.196479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5a1ff-c7a1-4bbf-a309-8ae68c999c10 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:14.196761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5a1ff-c7a1-4bbf-a309-8ae68c999c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:14.197023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5a1ff-c7a1-4bbf-a309-8ae68c999c10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:14.203922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5a1ff-c7a1-4bbf-a309-8ae68c999c10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:14.205843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5a1ff-c7a1-4bbf-a309-8ae68c999c10 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-fad5a1ff-c7a1-4bbf-a309-8ae68c999c10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:14.220332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:14.221863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:14.222735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:14.222735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:14.222735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:14.222735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:14.222735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:14.222735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:14.222735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:14.222735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:14.222735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 260/514] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:14.222735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5a1ff-c7a1-4bbf-a309-8ae68c999c10 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:14.234919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:14.241486 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c6729-957d-46f6-9ceb-c47da8cfe79a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:14.276247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360a8cf-307e-4357-96c1-b6ba5c9ee65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:14.276920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360a8cf-307e-4357-96c1-b6ba5c9ee65d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:14.277202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360a8cf-307e-4357-96c1-b6ba5c9ee65d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:14.277473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360a8cf-307e-4357-96c1-b6ba5c9ee65d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:14.333232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5a1ff-c7a1-4bbf-a309-8ae68c999c10 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:14.342320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:14.361367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360a8cf-307e-4357-96c1-b6ba5c9ee65d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:14.362838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360a8cf-307e-4357-96c1-b6ba5c9ee65d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:14.370582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52120773-89c0-47dd-a62c-7db2487a73fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:14.371538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52120773-89c0-47dd-a62c-7db2487a73fc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:14.371823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52120773-89c0-47dd-a62c-7db2487a73fc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:14.372100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52120773-89c0-47dd-a62c-7db2487a73fc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:14.380235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52120773-89c0-47dd-a62c-7db2487a73fc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:14.381723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52120773-89c0-47dd-a62c-7db2487a73fc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-52120773-89c0-47dd-a62c-7db2487a73fc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:14.395979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:14.397074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 261/515] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:14.398180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52120773-89c0-47dd-a62c-7db2487a73fc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:14.458700 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:14.466197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 255/516] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:14.467041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360a8cf-307e-4357-96c1-b6ba5c9ee65d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:14.498833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52120773-89c0-47dd-a62c-7db2487a73fc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:14.534670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136c376-d8ac-4353-9f6c-4f1883e1353c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:14.535344 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c376-d8ac-4353-9f6c-4f1883e1353c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:14.535626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c376-d8ac-4353-9f6c-4f1883e1353c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:14.535896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c376-d8ac-4353-9f6c-4f1883e1353c None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:14.536626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136c376-d8ac-4353-9f6c-4f1883e1353c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:14.536939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136c376-d8ac-4353-9f6c-4f1883e1353c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:14.538334 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136c376-d8ac-4353-9f6c-4f1883e1353c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:14.550524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 262/517] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:14.551379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136c376-d8ac-4353-9f6c-4f1883e1353c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:14.572665 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360a8cf-307e-4357-96c1-b6ba5c9ee65d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:14.579506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:14.610070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926868b-20d8-49db-b841-2541f3767867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:14.610678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926868b-20d8-49db-b841-2541f3767867 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:14.611105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926868b-20d8-49db-b841-2541f3767867 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:14.611494 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926868b-20d8-49db-b841-2541f3767867 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:14.612378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926868b-20d8-49db-b841-2541f3767867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:14.612694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926868b-20d8-49db-b841-2541f3767867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:14.614507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926868b-20d8-49db-b841-2541f3767867 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:14.627743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 256/518] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:14.628580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926868b-20d8-49db-b841-2541f3767867 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:14.658359 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136c376-d8ac-4353-9f6c-4f1883e1353c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:14.689260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:14.693585 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6a148-a7e6-4baa-b5e0-32b3b83ef145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:14.694250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6a148-a7e6-4baa-b5e0-32b3b83ef145 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:14.694531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6a148-a7e6-4baa-b5e0-32b3b83ef145 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:14.694827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6a148-a7e6-4baa-b5e0-32b3b83ef145 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:14.695612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6a148-a7e6-4baa-b5e0-32b3b83ef145 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:14.695916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6a148-a7e6-4baa-b5e0-32b3b83ef145 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:14.697367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6a148-a7e6-4baa-b5e0-32b3b83ef145 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:14.715370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 263/519] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:14.716195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6a148-a7e6-4baa-b5e0-32b3b83ef145 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:14.741581 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926868b-20d8-49db-b841-2541f3767867 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:14.776519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dfbf6-ff09-4fae-a292-ef79ac555323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:14.777195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dfbf6-ff09-4fae-a292-ef79ac555323 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:14.777488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dfbf6-ff09-4fae-a292-ef79ac555323 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:14.777753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dfbf6-ff09-4fae-a292-ef79ac555323 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:14.778640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dfbf6-ff09-4fae-a292-ef79ac555323 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:14.778953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dfbf6-ff09-4fae-a292-ef79ac555323 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:14.780406 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dfbf6-ff09-4fae-a292-ef79ac555323 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:14.809620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:14.817563 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 257/520] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 09:45:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:14.818408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dfbf6-ff09-4fae-a292-ef79ac555323 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:14.823807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6a148-a7e6-4baa-b5e0-32b3b83ef145 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:14.860337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a03ae-c39d-45b0-8caf-c2a575e2275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:14.861150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a03ae-c39d-45b0-8caf-c2a575e2275e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:14.861452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a03ae-c39d-45b0-8caf-c2a575e2275e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:14.861771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a03ae-c39d-45b0-8caf-c2a575e2275e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:14.872284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a03ae-c39d-45b0-8caf-c2a575e2275e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:14.873761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a03ae-c39d-45b0-8caf-c2a575e2275e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-5a8a03ae-c39d-45b0-8caf-c2a575e2275e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:14.892751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:14.893812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:14.894951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:14.894951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:14.894951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:14.894951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:14.894951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 264/521] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:14.895381 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a03ae-c39d-45b0-8caf-c2a575e2275e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:14.924412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dfbf6-ff09-4fae-a292-ef79ac555323 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:14.939717 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:14.960829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87417aba-da6e-4272-93ec-61ceaa6440a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:14.961644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87417aba-da6e-4272-93ec-61ceaa6440a9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:14.961924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87417aba-da6e-4272-93ec-61ceaa6440a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:14.962203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87417aba-da6e-4272-93ec-61ceaa6440a9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:14.969657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87417aba-da6e-4272-93ec-61ceaa6440a9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:14.971236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87417aba-da6e-4272-93ec-61ceaa6440a9 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-87417aba-da6e-4272-93ec-61ceaa6440a9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:14.988187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:14.989252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 258/522] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:14.990325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87417aba-da6e-4272-93ec-61ceaa6440a9 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:15.008285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a03ae-c39d-45b0-8caf-c2a575e2275e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:15.044858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523ba7b-81d1-46c9-8aa5-c4ca690b45a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:15.045539 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523ba7b-81d1-46c9-8aa5-c4ca690b45a3 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:15.045824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523ba7b-81d1-46c9-8aa5-c4ca690b45a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:15.046100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523ba7b-81d1-46c9-8aa5-c4ca690b45a3 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:15.046844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523ba7b-81d1-46c9-8aa5-c4ca690b45a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:15.048467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523ba7b-81d1-46c9-8aa5-c4ca690b45a3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:15.058514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:15.074757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 265/523] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:45:14 2022] POST /identity/v3/users => generated 247 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:45:15.075767 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523ba7b-81d1-46c9-8aa5-c4ca690b45a3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:15.099138 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87417aba-da6e-4272-93ec-61ceaa6440a9 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:15.138653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d271398-16cb-4d82-aafd-cee0f77c41ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:15.139453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d271398-16cb-4d82-aafd-cee0f77c41ff None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:15.139653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d271398-16cb-4d82-aafd-cee0f77c41ff None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:15.139966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d271398-16cb-4d82-aafd-cee0f77c41ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:15.140830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d271398-16cb-4d82-aafd-cee0f77c41ff None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=8a5eea7f539048c48ed10e72dd933df9) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:15.141144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d271398-16cb-4d82-aafd-cee0f77c41ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:15.142807 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d271398-16cb-4d82-aafd-cee0f77c41ff None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:15.149430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 259/524] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:14 2022] GET /identity/v3/role_assignments?user.id=8a5eea7f539048c48ed10e72dd933df9&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:15.150310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d271398-16cb-4d82-aafd-cee0f77c41ff None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:15.207372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523ba7b-81d1-46c9-8aa5-c4ca690b45a3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:15.216533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:15.258686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517ea7d-24c9-47e7-a9eb-9ce2b7788abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:15.259690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517ea7d-24c9-47e7-a9eb-9ce2b7788abc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:15.260528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517ea7d-24c9-47e7-a9eb-9ce2b7788abc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:15.260528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517ea7d-24c9-47e7-a9eb-9ce2b7788abc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:15.310189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d271398-16cb-4d82-aafd-cee0f77c41ff None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:15.339961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:15.347656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517ea7d-24c9-47e7-a9eb-9ce2b7788abc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:15.350118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517ea7d-24c9-47e7-a9eb-9ce2b7788abc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:15.356857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce7738-66d5-4c88-8a01-496e3d7b0332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:15.357542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7738-66d5-4c88-8a01-496e3d7b0332 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:15.357822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7738-66d5-4c88-8a01-496e3d7b0332 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:15.358095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7738-66d5-4c88-8a01-496e3d7b0332 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:15.358888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce7738-66d5-4c88-8a01-496e3d7b0332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:15.359286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce7738-66d5-4c88-8a01-496e3d7b0332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:15.360786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce7738-66d5-4c88-8a01-496e3d7b0332 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:15.374123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 260/525] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:15.375404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce7738-66d5-4c88-8a01-496e3d7b0332 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:15.478857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 266/526] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:15.480153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517ea7d-24c9-47e7-a9eb-9ce2b7788abc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:15.504672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:15.509194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce7738-66d5-4c88-8a01-496e3d7b0332 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:15.556181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adb4f2-60f3-4bc2-b5a0-fbd245b86e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:15.557276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adb4f2-60f3-4bc2-b5a0-fbd245b86e59 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:15.557560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adb4f2-60f3-4bc2-b5a0-fbd245b86e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:15.557837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adb4f2-60f3-4bc2-b5a0-fbd245b86e59 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:15.558986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02adb4f2-60f3-4bc2-b5a0-fbd245b86e59 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:15.559296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02adb4f2-60f3-4bc2-b5a0-fbd245b86e59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:15.561366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02adb4f2-60f3-4bc2-b5a0-fbd245b86e59 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:15.586659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 261/527] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Jun 29 09:45:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:15.587895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adb4f2-60f3-4bc2-b5a0-fbd245b86e59 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:15.609005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:15.616473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517ea7d-24c9-47e7-a9eb-9ce2b7788abc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:15.660478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02b9d8-cd40-48f2-ba82-edbed97449e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:15.661249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02b9d8-cd40-48f2-ba82-edbed97449e4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:15.661527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02b9d8-cd40-48f2-ba82-edbed97449e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:15.661798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02b9d8-cd40-48f2-ba82-edbed97449e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:15.670151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02b9d8-cd40-48f2-ba82-edbed97449e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:15.672533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02b9d8-cd40-48f2-ba82-edbed97449e4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-cb02b9d8-cd40-48f2-ba82-edbed97449e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:15.691282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:15.692185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:15.693110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:15.693110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:15.693110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:15.693110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:15.693110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:15.693110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:15.693110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:15.693110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:15.693110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 267/528] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:15.694020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02b9d8-cd40-48f2-ba82-edbed97449e4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:15.715036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adb4f2-60f3-4bc2-b5a0-fbd245b86e59 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:15.753235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f775f5d-2bee-4c11-9ae4-5e3b24605d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:15.754432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f775f5d-2bee-4c11-9ae4-5e3b24605d63 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:15.754830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f775f5d-2bee-4c11-9ae4-5e3b24605d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:15.755201 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f775f5d-2bee-4c11-9ae4-5e3b24605d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:15.768710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f775f5d-2bee-4c11-9ae4-5e3b24605d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:15.771109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f775f5d-2bee-4c11-9ae4-5e3b24605d63 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-1f775f5d-2bee-4c11-9ae4-5e3b24605d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:15.790128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:15.791256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:15.792307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:15.792307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:15.792307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:15.792307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:15.798267 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 262/529] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:15.798267 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f775f5d-2bee-4c11-9ae4-5e3b24605d63 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:15.808086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:15.837515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02b9d8-cd40-48f2-ba82-edbed97449e4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:15.887604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961918c-ab2a-4be1-8e7e-4d2efa90286a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:15.888807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961918c-ab2a-4be1-8e7e-4d2efa90286a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:15.889252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961918c-ab2a-4be1-8e7e-4d2efa90286a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:15.890122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961918c-ab2a-4be1-8e7e-4d2efa90286a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:15.909405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f775f5d-2bee-4c11-9ae4-5e3b24605d63 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:15.926840 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:15.947713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e534-24e9-4a0a-9b8b-9839dd2a7d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:15.948591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e534-24e9-4a0a-9b8b-9839dd2a7d12 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:15.948967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e534-24e9-4a0a-9b8b-9839dd2a7d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:15.949386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e534-24e9-4a0a-9b8b-9839dd2a7d12 None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:15.950421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4e534-24e9-4a0a-9b8b-9839dd2a7d12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:15.950768 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4e534-24e9-4a0a-9b8b-9839dd2a7d12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:15.952329 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4e534-24e9-4a0a-9b8b-9839dd2a7d12 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:15.972169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 263/530] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:15.973167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e534-24e9-4a0a-9b8b-9839dd2a7d12 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:15.983343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961918c-ab2a-4be1-8e7e-4d2efa90286a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:15.985505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961918c-ab2a-4be1-8e7e-4d2efa90286a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:16.053731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:16.100283 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e534-24e9-4a0a-9b8b-9839dd2a7d12 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:16.112847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 268/531] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:16.112847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961918c-ab2a-4be1-8e7e-4d2efa90286a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:16.148668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de703a4c-1f92-43e6-af92-5ca33a622355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:16.149981 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de703a4c-1f92-43e6-af92-5ca33a622355 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:16.150414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de703a4c-1f92-43e6-af92-5ca33a622355 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:16.150843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de703a4c-1f92-43e6-af92-5ca33a622355 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:16.151710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de703a4c-1f92-43e6-af92-5ca33a622355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:16.152173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de703a4c-1f92-43e6-af92-5ca33a622355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:16.153755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de703a4c-1f92-43e6-af92-5ca33a622355 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:16.174445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 264/532] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:16.174445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de703a4c-1f92-43e6-af92-5ca33a622355 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:16.175735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:16.222846 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961918c-ab2a-4be1-8e7e-4d2efa90286a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:16.268117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9e1e7-e3f4-456e-9bf6-9bcb5cb8da96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:16.269498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9e1e7-e3f4-456e-9bf6-9bcb5cb8da96 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:16.270004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9e1e7-e3f4-456e-9bf6-9bcb5cb8da96 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:16.270479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9e1e7-e3f4-456e-9bf6-9bcb5cb8da96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:16.272977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9e1e7-e3f4-456e-9bf6-9bcb5cb8da96 None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=8e850be24df24bcf972e6e31002014cb) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:16.273619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9e1e7-e3f4-456e-9bf6-9bcb5cb8da96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:16.275393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9e1e7-e3f4-456e-9bf6-9bcb5cb8da96 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:16.290612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 269/533] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:16 2022] GET /identity/v3/role_assignments?user.id=8e850be24df24bcf972e6e31002014cb&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:16.290612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9e1e7-e3f4-456e-9bf6-9bcb5cb8da96 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:16.336344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de703a4c-1f92-43e6-af92-5ca33a622355 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:16.391730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:16.401177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45671f63-ec87-4f39-b5ea-f66125ae16bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:16.401999 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45671f63-ec87-4f39-b5ea-f66125ae16bf None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:16.402281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45671f63-ec87-4f39-b5ea-f66125ae16bf None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:16.402556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45671f63-ec87-4f39-b5ea-f66125ae16bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:16.416418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45671f63-ec87-4f39-b5ea-f66125ae16bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:16.418156 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45671f63-ec87-4f39-b5ea-f66125ae16bf None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-45671f63-ec87-4f39-b5ea-f66125ae16bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:16.460256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:16.461173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:16.467070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:16.467070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:16.467070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:16.467070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:16.467070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 265/534] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:16.472366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9e1e7-e3f4-456e-9bf6-9bcb5cb8da96 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:16.484101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45671f63-ec87-4f39-b5ea-f66125ae16bf None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:16.497688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:16.515030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e13e1-5519-4efa-87bd-1f465e99c833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:16.516258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e13e1-5519-4efa-87bd-1f465e99c833 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:16.516768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e13e1-5519-4efa-87bd-1f465e99c833 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:16.517485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e13e1-5519-4efa-87bd-1f465e99c833 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:16.526154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e13e1-5519-4efa-87bd-1f465e99c833 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:16.528148 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e13e1-5519-4efa-87bd-1f465e99c833 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-489e13e1-5519-4efa-87bd-1f465e99c833 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:16.545927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:16.546829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:16.547681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:16.547681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:16.547681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:16.547681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:16.547681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:16.547681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:16.547681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:16.547681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:16.554780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 270/535] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:16.556575 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e13e1-5519-4efa-87bd-1f465e99c833 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:16.663955 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e13e1-5519-4efa-87bd-1f465e99c833 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:16.700206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668dab12-281f-405c-9665-a9c5cf15f463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:16.701235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668dab12-281f-405c-9665-a9c5cf15f463 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:16.701735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668dab12-281f-405c-9665-a9c5cf15f463 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:16.702219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668dab12-281f-405c-9665-a9c5cf15f463 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:16.703245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668dab12-281f-405c-9665-a9c5cf15f463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:16.703751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668dab12-281f-405c-9665-a9c5cf15f463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:16.705626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668dab12-281f-405c-9665-a9c5cf15f463 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:16.724581 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 271/536] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:16.726333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668dab12-281f-405c-9665-a9c5cf15f463 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:16.746695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45671f63-ec87-4f39-b5ea-f66125ae16bf None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:16.771840 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:16.806587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7120dab-1475-4f31-b236-bc6734779925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:16.806587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7120dab-1475-4f31-b236-bc6734779925 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:16.806587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7120dab-1475-4f31-b236-bc6734779925 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:16.806587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7120dab-1475-4f31-b236-bc6734779925 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:16.807002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7120dab-1475-4f31-b236-bc6734779925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:16.807002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7120dab-1475-4f31-b236-bc6734779925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:16.807130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7120dab-1475-4f31-b236-bc6734779925 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:16.837185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 266/537] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:16.842477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7120dab-1475-4f31-b236-bc6734779925 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:16.865980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668dab12-281f-405c-9665-a9c5cf15f463 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:16.923014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:16.927897 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c1a3c-9c35-4770-95f6-457e0514828c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:16.928612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c1a3c-9c35-4770-95f6-457e0514828c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:16.928899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c1a3c-9c35-4770-95f6-457e0514828c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:16.929174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c1a3c-9c35-4770-95f6-457e0514828c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:16.930017 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c1a3c-9c35-4770-95f6-457e0514828c None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=34813be7e8fd4fcfb5907bee6a465d16) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:16.930331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c1a3c-9c35-4770-95f6-457e0514828c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:16.931844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c1a3c-9c35-4770-95f6-457e0514828c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:16.946410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 272/538] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:16 2022] GET /identity/v3/role_assignments?user.id=34813be7e8fd4fcfb5907bee6a465d16&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:17.018241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7120dab-1475-4f31-b236-bc6734779925 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:17.098438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b429518-edf0-4cbc-90cb-55a89f74164a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:17.099677 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b429518-edf0-4cbc-90cb-55a89f74164a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:17.099973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b429518-edf0-4cbc-90cb-55a89f74164a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:17.100241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b429518-edf0-4cbc-90cb-55a89f74164a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:17.108240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b429518-edf0-4cbc-90cb-55a89f74164a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:17.116211 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b429518-edf0-4cbc-90cb-55a89f74164a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-9b429518-edf0-4cbc-90cb-55a89f74164a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:17.134676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:17.135584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:17.136426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:17.136426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:17.136426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:17.136426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:17.136426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:17.136426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:17.136426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:17.136426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:17.142684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 267/539] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:17.145577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c1a3c-9c35-4770-95f6-457e0514828c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:17.274728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c1a3c-9c35-4770-95f6-457e0514828c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:17.277496 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:17.327387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf091e4-b994-468b-bb68-8e7abb7eb16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:17.328101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf091e4-b994-468b-bb68-8e7abb7eb16c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:17.328389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf091e4-b994-468b-bb68-8e7abb7eb16c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:17.328672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf091e4-b994-468b-bb68-8e7abb7eb16c None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:17.329574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf091e4-b994-468b-bb68-8e7abb7eb16c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:17.329913 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf091e4-b994-468b-bb68-8e7abb7eb16c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:17.331516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf091e4-b994-468b-bb68-8e7abb7eb16c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:17.348293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 273/540] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:17.355558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b429518-edf0-4cbc-90cb-55a89f74164a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:17.433707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c3648-c735-4e18-a108-ff350012f053 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:17.434050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c3648-c735-4e18-a108-ff350012f053 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:17.434269 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c3648-c735-4e18-a108-ff350012f053 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:17.436067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 274/541] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:17.441637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab673278-2833-4d69-abe4-c9b5916ccc92 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:17.441908 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab673278-2833-4d69-abe4-c9b5916ccc92 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:17.442191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab673278-2833-4d69-abe4-c9b5916ccc92 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:17.443884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 275/542] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:17.444990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e8891-9a78-4927-9307-ce8820717c93 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:17.445372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e8891-9a78-4927-9307-ce8820717c93 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:17.445550 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e8891-9a78-4927-9307-ce8820717c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:17.463002 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3e9e8891-9a78-4927-9307-ce8820717c93 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:17.464191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3e9e8891-9a78-4927-9307-ce8820717c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:17.518834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:17.523798 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b429518-edf0-4cbc-90cb-55a89f74164a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:17.557075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 276/543] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:17.559388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc347-4555-4d77-8282-04c81377976d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:17.559814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc347-4555-4d77-8282-04c81377976d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:17.560228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc347-4555-4d77-8282-04c81377976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:17.561881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e8a94-cf2c-4159-87a1-d0370dd2dc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:17.562694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e8a94-cf2c-4159-87a1-d0370dd2dc58 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:17.563062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e8a94-cf2c-4159-87a1-d0370dd2dc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:17.563431 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e8a94-cf2c-4159-87a1-d0370dd2dc58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:17.564704 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e8a94-cf2c-4159-87a1-d0370dd2dc58 None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=4c97b321bf854177bd2f310cf2958959) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:17.565178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e8a94-cf2c-4159-87a1-d0370dd2dc58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:17.566925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e8a94-cf2c-4159-87a1-d0370dd2dc58 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:17.582604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 268/544] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:17 2022] GET /identity/v3/role_assignments?user.id=4c97b321bf854177bd2f310cf2958959&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:17.592041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-466dc347-4555-4d77-8282-04c81377976d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:17.593363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-466dc347-4555-4d77-8282-04c81377976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:17.598317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e8a94-cf2c-4159-87a1-d0370dd2dc58 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:17.737877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 277/545] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:17.768225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466dc347-4555-4d77-8282-04c81377976d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:17.773226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e8a94-cf2c-4159-87a1-d0370dd2dc58 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:17.797503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:17.813595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0f324-519a-455c-8e4b-90a6b37be1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:17.814826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0f324-519a-455c-8e4b-90a6b37be1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:17.815326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0f324-519a-455c-8e4b-90a6b37be1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:17.815806 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0f324-519a-455c-8e4b-90a6b37be1e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:17.827724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0f324-519a-455c-8e4b-90a6b37be1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-17b0f324-519a-455c-8e4b-90a6b37be1e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:17.836442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:17.837325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:17.838118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:17.840850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 269/546] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:17.847623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0f324-519a-455c-8e4b-90a6b37be1e6 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:17.876788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:17.944585 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466dc347-4555-4d77-8282-04c81377976d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:18.015467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883836d-4813-4022-a467-6be7d0387253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:18.016828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883836d-4813-4022-a467-6be7d0387253 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:18.019071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0f324-519a-455c-8e4b-90a6b37be1e6 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:18.019647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883836d-4813-4022-a467-6be7d0387253 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:18.020149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883836d-4813-4022-a467-6be7d0387253 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:18.059258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f883836d-4813-4022-a467-6be7d0387253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f883836d-4813-4022-a467-6be7d0387253 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:18.068066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:18.069451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:18.071436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 278/547] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:18.083829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883836d-4813-4022-a467-6be7d0387253 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:18.121089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffc4fe-e3ed-485b-914c-9eae97d29139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:18.121779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc4fe-e3ed-485b-914c-9eae97d29139 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:18.122062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc4fe-e3ed-485b-914c-9eae97d29139 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:18.122339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc4fe-e3ed-485b-914c-9eae97d29139 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:18.123127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffc4fe-e3ed-485b-914c-9eae97d29139 None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:18.123439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffc4fe-e3ed-485b-914c-9eae97d29139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:18.124836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffc4fe-e3ed-485b-914c-9eae97d29139 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:18.143761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 270/548] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:18.147686 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffc4fe-e3ed-485b-914c-9eae97d29139 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:18.259073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:18.276678 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883836d-4813-4022-a467-6be7d0387253 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:18.315725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:18.334354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd701a-8699-44d5-81cf-6a8fc34afa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:18.334989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd701a-8699-44d5-81cf-6a8fc34afa8a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:18.335263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd701a-8699-44d5-81cf-6a8fc34afa8a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:18.335542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd701a-8699-44d5-81cf-6a8fc34afa8a None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:18.336309 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd701a-8699-44d5-81cf-6a8fc34afa8a None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:18.337003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd701a-8699-44d5-81cf-6a8fc34afa8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:18.338773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd701a-8699-44d5-81cf-6a8fc34afa8a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:18.346075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffc4fe-e3ed-485b-914c-9eae97d29139 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:18.354369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 279/549] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:18.365028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd701a-8699-44d5-81cf-6a8fc34afa8a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:18.422421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15128640-6e69-4f49-8f65-3158298978d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:18.423659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15128640-6e69-4f49-8f65-3158298978d5 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:18.423944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15128640-6e69-4f49-8f65-3158298978d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:18.424216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15128640-6e69-4f49-8f65-3158298978d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:18.449895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15128640-6e69-4f49-8f65-3158298978d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:18.451412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15128640-6e69-4f49-8f65-3158298978d5 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-15128640-6e69-4f49-8f65-3158298978d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:18.484544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:18.486016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:18.487154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:18.487154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:18.487154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:18.487154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:18.487787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 271/550] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:18.496187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15128640-6e69-4f49-8f65-3158298978d5 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:18.578986 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd701a-8699-44d5-81cf-6a8fc34afa8a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:18.617984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d5a89-d126-46b9-852a-a924c3cd8083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:18.619182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5a89-d126-46b9-852a-a924c3cd8083 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:18.619661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5a89-d126-46b9-852a-a924c3cd8083 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:18.620125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5a89-d126-46b9-852a-a924c3cd8083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:18.631534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d5a89-d126-46b9-852a-a924c3cd8083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:18.633308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d5a89-d126-46b9-852a-a924c3cd8083 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:18.640459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:18.648160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15128640-6e69-4f49-8f65-3158298978d5 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-027d5a89-d126-46b9-852a-a924c3cd8083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:18.654171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:18.655257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:18.656497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:18.656497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:18.656497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:18.656497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:18.657953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 280/551] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:18.659480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bca0fc-8903-4255-8c0c-c1b17c4e2525 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:18.659779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bca0fc-8903-4255-8c0c-c1b17c4e2525 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:18.660141 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bca0fc-8903-4255-8c0c-c1b17c4e2525 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:18.665900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 281/552] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:18.665900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bca0fc-8903-4255-8c0c-c1b17c4e2525 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:18.701530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d90dbc-fe0f-4852-a33b-fe75846df689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:18.702344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90dbc-fe0f-4852-a33b-fe75846df689 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:18.702776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90dbc-fe0f-4852-a33b-fe75846df689 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:18.703059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90dbc-fe0f-4852-a33b-fe75846df689 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:18.703793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d90dbc-fe0f-4852-a33b-fe75846df689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:18.704102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d90dbc-fe0f-4852-a33b-fe75846df689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:18.706623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d90dbc-fe0f-4852-a33b-fe75846df689 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:18.717941 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:18.732591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 272/553] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:18.733969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715f5e6-8dd9-4015-b34e-949da012506c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:18.734256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715f5e6-8dd9-4015-b34e-949da012506c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:18.734534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715f5e6-8dd9-4015-b34e-949da012506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:18.753055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-5715f5e6-8dd9-4015-b34e-949da012506c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:18.754816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5715f5e6-8dd9-4015-b34e-949da012506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:18.802796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bca0fc-8903-4255-8c0c-c1b17c4e2525 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:18.846686 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 273/554] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:18.846686 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715f5e6-8dd9-4015-b34e-949da012506c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:18.852837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75025eb7-f553-4571-8774-685857a4e977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:18.853861 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75025eb7-f553-4571-8774-685857a4e977 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:18.854340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75025eb7-f553-4571-8774-685857a4e977 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:18.854881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75025eb7-f553-4571-8774-685857a4e977 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:18.855832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75025eb7-f553-4571-8774-685857a4e977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:18.856336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75025eb7-f553-4571-8774-685857a4e977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:18.858071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75025eb7-f553-4571-8774-685857a4e977 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:18.880768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 282/555] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:18.886635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75025eb7-f553-4571-8774-685857a4e977 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:18.950782 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715f5e6-8dd9-4015-b34e-949da012506c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:18.951416 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:18.985516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:18.995613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b73a0-634c-4c4c-a829-3e81bba76073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:18.996826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b73a0-634c-4c4c-a829-3e81bba76073 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:18.997414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b73a0-634c-4c4c-a829-3e81bba76073 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:18.997995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b73a0-634c-4c4c-a829-3e81bba76073 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.001779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75025eb7-f553-4571-8774-685857a4e977 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:19.008631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b73a0-634c-4c4c-a829-3e81bba76073 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:19.010433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b73a0-634c-4c4c-a829-3e81bba76073 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-5a7b73a0-634c-4c4c-a829-3e81bba76073 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:19.028301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:19.029289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:19.030198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:19.030198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:19.030198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:19.030198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:19.030198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:19.030198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:19.030198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:19.030198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:19.030198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:19.030198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:19.033034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 274/556] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:19.034090 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b73a0-634c-4c4c-a829-3e81bba76073 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:19.058496 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec490e6-870a-4ede-be8a-607a28df3667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:19.059348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec490e6-870a-4ede-be8a-607a28df3667 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.059632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec490e6-870a-4ede-be8a-607a28df3667 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.059905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec490e6-870a-4ede-be8a-607a28df3667 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.073641 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec490e6-870a-4ede-be8a-607a28df3667 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:19.075706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec490e6-870a-4ede-be8a-607a28df3667 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-aec490e6-870a-4ede-be8a-607a28df3667 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:19.094945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:19.096136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:19.097526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:19.097526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:19.097526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:19.097526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:19.097526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:19.097526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:19.097526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:19.097526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:19.097526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:19.097526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:19.100524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 283/557] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 09:45:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:19.101183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec490e6-870a-4ede-be8a-607a28df3667 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:19.152743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b73a0-634c-4c4c-a829-3e81bba76073 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:19.188243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c837c-f3d0-4fee-9e1b-75bdedc5110a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:19.189476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c837c-f3d0-4fee-9e1b-75bdedc5110a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.189958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c837c-f3d0-4fee-9e1b-75bdedc5110a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.190425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c837c-f3d0-4fee-9e1b-75bdedc5110a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.201392 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c837c-f3d0-4fee-9e1b-75bdedc5110a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-465c837c-f3d0-4fee-9e1b-75bdedc5110a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:19.209574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:19.210484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:19.211371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:19.213684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec490e6-870a-4ede-be8a-607a28df3667 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:19.214358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 275/558] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:19.215463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c837c-f3d0-4fee-9e1b-75bdedc5110a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:19.234953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:19.254303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f7e95-b093-428e-b1f1-acef3f01e21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:19.255230 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f7e95-b093-428e-b1f1-acef3f01e21f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.255610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f7e95-b093-428e-b1f1-acef3f01e21f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.256027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f7e95-b093-428e-b1f1-acef3f01e21f None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.257142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f7e95-b093-428e-b1f1-acef3f01e21f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:19.257553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f7e95-b093-428e-b1f1-acef3f01e21f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:19.259254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f7e95-b093-428e-b1f1-acef3f01e21f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:19.272917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:19.284021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 284/559] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 09:45:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:19.284021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f7e95-b093-428e-b1f1-acef3f01e21f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:19.344727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c837c-f3d0-4fee-9e1b-75bdedc5110a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:19.388613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2e0a5-6c21-44b4-87b2-e35588e5c512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:19.389414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2e0a5-6c21-44b4-87b2-e35588e5c512 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.389764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2e0a5-6c21-44b4-87b2-e35588e5c512 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.390115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2e0a5-6c21-44b4-87b2-e35588e5c512 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.391059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2e0a5-6c21-44b4-87b2-e35588e5c512 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:19.391462 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2e0a5-6c21-44b4-87b2-e35588e5c512 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:19.393139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2e0a5-6c21-44b4-87b2-e35588e5c512 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:19.410030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f7e95-b093-428e-b1f1-acef3f01e21f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:19.415398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 276/560] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 09:45:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:19.415398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2e0a5-6c21-44b4-87b2-e35588e5c512 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:19.449333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cc1c1-d667-431f-ac6f-c0354df4ea10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:19.450015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cc1c1-d667-431f-ac6f-c0354df4ea10 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.450298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cc1c1-d667-431f-ac6f-c0354df4ea10 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.450584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cc1c1-d667-431f-ac6f-c0354df4ea10 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.451355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cc1c1-d667-431f-ac6f-c0354df4ea10 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:19.451664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cc1c1-d667-431f-ac6f-c0354df4ea10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:19.453066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cc1c1-d667-431f-ac6f-c0354df4ea10 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:19.466959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 285/561] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:19.466959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d68d6f-421a-45ab-964d-431c92d61e58 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.466959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d68d6f-421a-45ab-964d-431c92d61e58 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.466959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d68d6f-421a-45ab-964d-431c92d61e58 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.466959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 286/562] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:19.466959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d68d6f-421a-45ab-964d-431c92d61e58 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:19.515112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:19.531067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2e0a5-6c21-44b4-87b2-e35588e5c512 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:19.539241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:19.567405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2b281-03b8-4d65-99a6-6084df2c104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:19.568142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2b281-03b8-4d65-99a6-6084df2c104c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.568425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2b281-03b8-4d65-99a6-6084df2c104c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.568736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2b281-03b8-4d65-99a6-6084df2c104c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.586765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d68d6f-421a-45ab-964d-431c92d61e58 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:19.625270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b949a-d520-4ee4-a524-09462b4f3f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:19.626788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b949a-d520-4ee4-a524-09462b4f3f6b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.627289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b949a-d520-4ee4-a524-09462b4f3f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.627634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b949a-d520-4ee4-a524-09462b4f3f6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.638637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b949a-d520-4ee4-a524-09462b4f3f6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:19.639731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2b281-03b8-4d65-99a6-6084df2c104c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:19.640180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b949a-d520-4ee4-a524-09462b4f3f6b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:19.641182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2b281-03b8-4d65-99a6-6084df2c104c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-947b949a-d520-4ee4-a524-09462b4f3f6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:19.659155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:19.660205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:19.661111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:19.661111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:19.661111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:19.661111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:19.661111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 287/563] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:19.661416 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b949a-d520-4ee4-a524-09462b4f3f6b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:19.758243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 277/564] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:19.760007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b4d75-d485-4282-9c88-529fa0a51b60 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.760296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b4d75-d485-4282-9c88-529fa0a51b60 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.760570 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b4d75-d485-4282-9c88-529fa0a51b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.764948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:19.772445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:19.777547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b949a-d520-4ee4-a524-09462b4f3f6b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:19.778368 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a40b4d75-d485-4282-9c88-529fa0a51b60 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:19.779592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a40b4d75-d485-4282-9c88-529fa0a51b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:19.815010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21dd69-77ce-480c-8aab-8a0b79eb377d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:19.815735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21dd69-77ce-480c-8aab-8a0b79eb377d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.816016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21dd69-77ce-480c-8aab-8a0b79eb377d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.816977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21dd69-77ce-480c-8aab-8a0b79eb377d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.827688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21dd69-77ce-480c-8aab-8a0b79eb377d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:19.829304 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21dd69-77ce-480c-8aab-8a0b79eb377d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-ed21dd69-77ce-480c-8aab-8a0b79eb377d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:19.849013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:19.850326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:19.851644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:19.851644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:19.851644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:19.851644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:19.851644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 288/565] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:19.851644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21dd69-77ce-480c-8aab-8a0b79eb377d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:19.862660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 278/566] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:19.862660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b4d75-d485-4282-9c88-529fa0a51b60 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:19.960394 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b4d75-d485-4282-9c88-529fa0a51b60 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:19.960871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21dd69-77ce-480c-8aab-8a0b79eb377d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:19.994966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960937f-9b3d-4c2c-9879-f1c177d8d360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:19.995240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bc647-e9f7-4ded-aa64-d96aaf41274b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:19.995765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960937f-9b3d-4c2c-9879-f1c177d8d360 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.995893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bc647-e9f7-4ded-aa64-d96aaf41274b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:19.996044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960937f-9b3d-4c2c-9879-f1c177d8d360 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.996168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bc647-e9f7-4ded-aa64-d96aaf41274b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:19.996315 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960937f-9b3d-4c2c-9879-f1c177d8d360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.996419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bc647-e9f7-4ded-aa64-d96aaf41274b None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:19.997165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bc647-e9f7-4ded-aa64-d96aaf41274b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:19.997453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bc647-e9f7-4ded-aa64-d96aaf41274b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:19.999122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bc647-e9f7-4ded-aa64-d96aaf41274b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:19.999252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:20.007578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960937f-9b3d-4c2c-9879-f1c177d8d360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:20.009017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960937f-9b3d-4c2c-9879-f1c177d8d360 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:20.011323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 289/567] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:20.012142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bc647-e9f7-4ded-aa64-d96aaf41274b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.014764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-3960937f-9b3d-4c2c-9879-f1c177d8d360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:20.032270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:20.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:20.034213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:20.034213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:20.034213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:20.034213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:20.034213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 279/568] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:20.034481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960937f-9b3d-4c2c-9879-f1c177d8d360 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.122545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bc647-e9f7-4ded-aa64-d96aaf41274b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:20.138016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960937f-9b3d-4c2c-9879-f1c177d8d360 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:20.159083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183882a3-dbef-4c9b-ad4c-6a9a7ea4fc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:20.159741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183882a3-dbef-4c9b-ad4c-6a9a7ea4fc6c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.160021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183882a3-dbef-4c9b-ad4c-6a9a7ea4fc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.160294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183882a3-dbef-4c9b-ad4c-6a9a7ea4fc6c None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.161021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183882a3-dbef-4c9b-ad4c-6a9a7ea4fc6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:20.161330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183882a3-dbef-4c9b-ad4c-6a9a7ea4fc6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:20.162768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183882a3-dbef-4c9b-ad4c-6a9a7ea4fc6c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:20.174226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33631fca-f66d-4b99-9709-4aa4fb9759c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:20.174410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 290/569] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:20.175065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33631fca-f66d-4b99-9709-4aa4fb9759c8 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.175340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33631fca-f66d-4b99-9709-4aa4fb9759c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.175588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265734a8-6597-4355-8ae3-6cc669bc84cf None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.175682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33631fca-f66d-4b99-9709-4aa4fb9759c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.175873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265734a8-6597-4355-8ae3-6cc669bc84cf None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.176191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265734a8-6597-4355-8ae3-6cc669bc84cf None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.180664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 291/570] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:20.180664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265734a8-6597-4355-8ae3-6cc669bc84cf None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.190231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33631fca-f66d-4b99-9709-4aa4fb9759c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-33631fca-f66d-4b99-9709-4aa4fb9759c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:20.198044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:20.199226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:20.200235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 280/571] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:20.201335 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33631fca-f66d-4b99-9709-4aa4fb9759c8 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.244312 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:20.252344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:20.284473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265734a8-6597-4355-8ae3-6cc669bc84cf None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:20.307327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33631fca-f66d-4b99-9709-4aa4fb9759c8 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:20.318905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5044b-7a0b-4a05-b31e-baf1b13cd4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:20.319706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5044b-7a0b-4a05-b31e-baf1b13cd4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.319980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5044b-7a0b-4a05-b31e-baf1b13cd4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.320240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5044b-7a0b-4a05-b31e-baf1b13cd4d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.327334 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5044b-7a0b-4a05-b31e-baf1b13cd4d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:20.328808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5044b-7a0b-4a05-b31e-baf1b13cd4d0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:20.341742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51905df-eea4-41d2-8a02-d75e979d28ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:20.342418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51905df-eea4-41d2-8a02-d75e979d28ec None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.342730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51905df-eea4-41d2-8a02-d75e979d28ec None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.343002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51905df-eea4-41d2-8a02-d75e979d28ec None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-19d5044b-7a0b-4a05-b31e-baf1b13cd4d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:20.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:20.344229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:20.345163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:20.345163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:20.345163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:20.345163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:20.345163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:20.345163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:20.345163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:20.345163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:20.345612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51905df-eea4-41d2-8a02-d75e979d28ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:20.345612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51905df-eea4-41d2-8a02-d75e979d28ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:20.345736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 292/572] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:20.345736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5044b-7a0b-4a05-b31e-baf1b13cd4d0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.345866 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51905df-eea4-41d2-8a02-d75e979d28ec None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:20.359987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 281/573] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:20.361345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe582c40-0920-4f58-ae31-901e658bf168 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.361716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe582c40-0920-4f58-ae31-901e658bf168 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.362073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe582c40-0920-4f58-ae31-901e658bf168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.378889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fe582c40-0920-4f58-ae31-901e658bf168 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:20.380163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fe582c40-0920-4f58-ae31-901e658bf168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:20.447190 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 282/574] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:20.448043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe582c40-0920-4f58-ae31-901e658bf168 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.454666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5044b-7a0b-4a05-b31e-baf1b13cd4d0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:20.478482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:20.490889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e023010-e63f-4272-a32d-96fa34e0d5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:20.491704 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e023010-e63f-4272-a32d-96fa34e0d5cb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.491984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e023010-e63f-4272-a32d-96fa34e0d5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.492259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e023010-e63f-4272-a32d-96fa34e0d5cb None admin] PATH_INFO: `/v3/users/glance` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.500643 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e023010-e63f-4272-a32d-96fa34e0d5cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:20.502175 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e023010-e63f-4272-a32d-96fa34e0d5cb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:20.503536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-6e023010-e63f-4272-a32d-96fa34e0d5cb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:20.517750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:20.518982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 293/575] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:20.519956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e023010-e63f-4272-a32d-96fa34e0d5cb None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.575059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe582c40-0920-4f58-ae31-901e658bf168 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:20.610213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ea697-9fc7-49eb-8afc-96b2a91599ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:20.610987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ea697-9fc7-49eb-8afc-96b2a91599ff None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.611247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ea697-9fc7-49eb-8afc-96b2a91599ff None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.611545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ea697-9fc7-49eb-8afc-96b2a91599ff None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.612304 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ea697-9fc7-49eb-8afc-96b2a91599ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:20.612615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ea697-9fc7-49eb-8afc-96b2a91599ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:20.614022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ea697-9fc7-49eb-8afc-96b2a91599ff None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:20.622706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 283/576] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:20.623537 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ea697-9fc7-49eb-8afc-96b2a91599ff None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.625173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e023010-e63f-4272-a32d-96fa34e0d5cb None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:20.661199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c9489-3bb8-427d-b78a-b3b4a13b7254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:20.661875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c9489-3bb8-427d-b78a-b3b4a13b7254 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.662158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c9489-3bb8-427d-b78a-b3b4a13b7254 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.662433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c9489-3bb8-427d-b78a-b3b4a13b7254 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.663277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c9489-3bb8-427d-b78a-b3b4a13b7254 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:20.663587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c9489-3bb8-427d-b78a-b3b4a13b7254 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:20.665039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c9489-3bb8-427d-b78a-b3b4a13b7254 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:20.677582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 294/577] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:20.678421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c9489-3bb8-427d-b78a-b3b4a13b7254 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.726916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:20.728413 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:20.732049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ea697-9fc7-49eb-8afc-96b2a91599ff None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:20.766188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d275ab3-8ec4-4d2e-a1f4-e0a2cd09c409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:20.767134 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d275ab3-8ec4-4d2e-a1f4-e0a2cd09c409 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.767390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d275ab3-8ec4-4d2e-a1f4-e0a2cd09c409 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.767630 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d275ab3-8ec4-4d2e-a1f4-e0a2cd09c409 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.775037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d275ab3-8ec4-4d2e-a1f4-e0a2cd09c409 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:20.776504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d275ab3-8ec4-4d2e-a1f4-e0a2cd09c409 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:20.787438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c9489-3bb8-427d-b78a-b3b4a13b7254 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-6d275ab3-8ec4-4d2e-a1f4-e0a2cd09c409 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:20.791483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:20.792515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:20.793522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:20.793522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:20.793522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:20.793522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:20.793522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:20.793522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:20.793522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:20.793522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:20.793522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 284/578] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:20.794153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d275ab3-8ec4-4d2e-a1f4-e0a2cd09c409 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.822532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b449c-5cad-430d-9989-277262ffd610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:20.823371 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b449c-5cad-430d-9989-277262ffd610 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.823650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b449c-5cad-430d-9989-277262ffd610 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.823925 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b449c-5cad-430d-9989-277262ffd610 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.834429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b449c-5cad-430d-9989-277262ffd610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-923b449c-5cad-430d-9989-277262ffd610 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:20.842182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:20.843374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:20.844378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 295/579] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:20.845153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b449c-5cad-430d-9989-277262ffd610 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.903353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d275ab3-8ec4-4d2e-a1f4-e0a2cd09c409 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:20.938394 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0f784-cd15-4cd8-9e89-5064223191bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:20.939233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0f784-cd15-4cd8-9e89-5064223191bd None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.939515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0f784-cd15-4cd8-9e89-5064223191bd None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.939793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0f784-cd15-4cd8-9e89-5064223191bd None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:20.940648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0f784-cd15-4cd8-9e89-5064223191bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:20.940963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0f784-cd15-4cd8-9e89-5064223191bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:20.942422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0f784-cd15-4cd8-9e89-5064223191bd None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:20.950220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b449c-5cad-430d-9989-277262ffd610 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:20.951825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:20.960618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 285/580] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 09:45:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:20.961720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0f784-cd15-4cd8-9e89-5064223191bd None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:20.981711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:20.991591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd21e0-ab9c-4624-9c4f-e7de70fdd1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:20.992464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd21e0-ab9c-4624-9c4f-e7de70fdd1ef None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:20.992746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd21e0-ab9c-4624-9c4f-e7de70fdd1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:20.993068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd21e0-ab9c-4624-9c4f-e7de70fdd1ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:21.004389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd21e0-ab9c-4624-9c4f-e7de70fdd1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:21.005889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd21e0-ab9c-4624-9c4f-e7de70fdd1ef None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-9abd21e0-ab9c-4624-9c4f-e7de70fdd1ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:21.023838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:21.024932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:21.025972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:21.025972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:21.025972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:21.025972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:21.025972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 296/581] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:21.026335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd21e0-ab9c-4624-9c4f-e7de70fdd1ef None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:21.076646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0f784-cd15-4cd8-9e89-5064223191bd None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:21.114881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b344a7f-9975-4de2-98f1-936b17df91c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:21.115943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344a7f-9975-4de2-98f1-936b17df91c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:21.116089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344a7f-9975-4de2-98f1-936b17df91c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:21.116271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344a7f-9975-4de2-98f1-936b17df91c1 None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/8a5eea7f539048c48ed10e72dd933df9/roles/9b6eb8ccaac643cc93781d6abe31bb8f` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:21.132942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b344a7f-9975-4de2-98f1-936b17df91c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=8a5eea7f539048c48ed10e72dd933df9, role_id=9b6eb8ccaac643cc93781d6abe31bb8f)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:21.134430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b344a7f-9975-4de2-98f1-936b17df91c1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:21.134805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd21e0-ab9c-4624-9c4f-e7de70fdd1ef None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:21.148480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 286/582] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:45:20 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/8a5eea7f539048c48ed10e72dd933df9/roles/9b6eb8ccaac643cc93781d6abe31bb8f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:45:21.149312 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b344a7f-9975-4de2-98f1-936b17df91c1 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:21.185311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892abde-9df5-437b-9000-b3f9e20c4b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:21.186054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892abde-9df5-437b-9000-b3f9e20c4b5e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:21.186340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892abde-9df5-437b-9000-b3f9e20c4b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:21.186649 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892abde-9df5-437b-9000-b3f9e20c4b5e None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:21.187491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892abde-9df5-437b-9000-b3f9e20c4b5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:21.187815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892abde-9df5-437b-9000-b3f9e20c4b5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:21.189690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892abde-9df5-437b-9000-b3f9e20c4b5e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:21.208435 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 297/583] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:21.208435 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892abde-9df5-437b-9000-b3f9e20c4b5e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:21.228752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:21.241219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:21.282972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b344a7f-9975-4de2-98f1-936b17df91c1 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:21.326832 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677f889-3d5e-41be-82fe-72f17bb0a325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:21.327621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677f889-3d5e-41be-82fe-72f17bb0a325 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:21.327976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677f889-3d5e-41be-82fe-72f17bb0a325 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:21.328324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677f889-3d5e-41be-82fe-72f17bb0a325 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:21.329160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677f889-3d5e-41be-82fe-72f17bb0a325 None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:21.329573 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677f889-3d5e-41be-82fe-72f17bb0a325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:21.331067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677f889-3d5e-41be-82fe-72f17bb0a325 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:21.336028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892abde-9df5-437b-9000-b3f9e20c4b5e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:21.340274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 287/584] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:21.341260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677f889-3d5e-41be-82fe-72f17bb0a325 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:21.374171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4d26d-c63c-4f02-8a44-eb241a3ee4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:21.374995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4d26d-c63c-4f02-8a44-eb241a3ee4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:21.375335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4d26d-c63c-4f02-8a44-eb241a3ee4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:21.375610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4d26d-c63c-4f02-8a44-eb241a3ee4e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:21.386114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4d26d-c63c-4f02-8a44-eb241a3ee4e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:21.387610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4d26d-c63c-4f02-8a44-eb241a3ee4e9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-82a4d26d-c63c-4f02-8a44-eb241a3ee4e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:21.405491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:21.406438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:21.407366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:21.407366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:21.407366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:21.407366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:21.407366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 298/585] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:21.407774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4d26d-c63c-4f02-8a44-eb241a3ee4e9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:21.460788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677f889-3d5e-41be-82fe-72f17bb0a325 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:21.475927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:21.502036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:21.510498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547af63-cdf3-4da1-9d61-df115d33fb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:21.511781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547af63-cdf3-4da1-9d61-df115d33fb94 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:21.512205 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547af63-cdf3-4da1-9d61-df115d33fb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:21.512476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547af63-cdf3-4da1-9d61-df115d33fb94 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:21.513210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547af63-cdf3-4da1-9d61-df115d33fb94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:21.513515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547af63-cdf3-4da1-9d61-df115d33fb94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:21.513672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4d26d-c63c-4f02-8a44-eb241a3ee4e9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:21.514941 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547af63-cdf3-4da1-9d61-df115d33fb94 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:21.532547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 288/586] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:21.532547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547af63-cdf3-4da1-9d61-df115d33fb94 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:21.560829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c381e1-a97b-4b85-916c-f88e71e73b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:21.561744 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c381e1-a97b-4b85-916c-f88e71e73b9b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:21.562174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c381e1-a97b-4b85-916c-f88e71e73b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:21.562699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c381e1-a97b-4b85-916c-f88e71e73b9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:21.563905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c381e1-a97b-4b85-916c-f88e71e73b9b None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=d86d073ef5ee4631b508ab578662be96) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:21.564231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c381e1-a97b-4b85-916c-f88e71e73b9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:21.565710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c381e1-a97b-4b85-916c-f88e71e73b9b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:21.582598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 299/587] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:21 2022] GET /identity/v3/role_assignments?user.id=d86d073ef5ee4631b508ab578662be96&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:21.582598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c381e1-a97b-4b85-916c-f88e71e73b9b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:21.657265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547af63-cdf3-4da1-9d61-df115d33fb94 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:21.699210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f61c19-5278-4d41-a811-d61e347b45c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:21.700297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f61c19-5278-4d41-a811-d61e347b45c6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:21.700637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f61c19-5278-4d41-a811-d61e347b45c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:21.700979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f61c19-5278-4d41-a811-d61e347b45c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:21.713416 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f61c19-5278-4d41-a811-d61e347b45c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:21.715071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f61c19-5278-4d41-a811-d61e347b45c6 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:21.738127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c381e1-a97b-4b85-916c-f88e71e73b9b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-b4f61c19-5278-4d41-a811-d61e347b45c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:21.740054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:21.740952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:21.741992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:21.741992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:21.741992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:21.741992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:21.741992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 289/588] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:21.743240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f61c19-5278-4d41-a811-d61e347b45c6 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:21.785512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:21.795236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:21.800428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de402a9-189a-4839-94db-3351795c0947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:21.801115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de402a9-189a-4839-94db-3351795c0947 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:21.801395 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de402a9-189a-4839-94db-3351795c0947 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:21.801664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de402a9-189a-4839-94db-3351795c0947 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:21.802409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de402a9-189a-4839-94db-3351795c0947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:21.802751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de402a9-189a-4839-94db-3351795c0947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:21.804168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de402a9-189a-4839-94db-3351795c0947 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:21.819934 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 300/589] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:21.821407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de402a9-189a-4839-94db-3351795c0947 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:21.865405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f61c19-5278-4d41-a811-d61e347b45c6 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:21.905611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c429adc-2164-4546-8ec1-4955f72a8b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:21.906876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429adc-2164-4546-8ec1-4955f72a8b52 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:21.907163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429adc-2164-4546-8ec1-4955f72a8b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:21.907725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429adc-2164-4546-8ec1-4955f72a8b52 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:21.918616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c429adc-2164-4546-8ec1-4955f72a8b52 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:21.921048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c429adc-2164-4546-8ec1-4955f72a8b52 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-0c429adc-2164-4546-8ec1-4955f72a8b52 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:21.943622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:21.944920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:21.946078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:21.946078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:21.946078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:21.946078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:21.946078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:21.946078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:21.946078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:21.946078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:21.946078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 09:45:21.946078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:21.952912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 290/590] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:21.952912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c429adc-2164-4546-8ec1-4955f72a8b52 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:21.967558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de402a9-189a-4839-94db-3351795c0947 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:22.014932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3f998-616b-418b-a1c7-7f9abaef1735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:22.015879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3f998-616b-418b-a1c7-7f9abaef1735 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:22.016305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3f998-616b-418b-a1c7-7f9abaef1735 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:22.017306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3f998-616b-418b-a1c7-7f9abaef1735 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:22.018446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3f998-616b-418b-a1c7-7f9abaef1735 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:22.019011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3f998-616b-418b-a1c7-7f9abaef1735 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:22.020659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3f998-616b-418b-a1c7-7f9abaef1735 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:22.034991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 301/591] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:22.036183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3f998-616b-418b-a1c7-7f9abaef1735 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:22.069907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c429adc-2164-4546-8ec1-4955f72a8b52 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:22.071953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:22.109649 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:22.111018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b0825-e9f0-45e3-b0ef-bf7d670cf294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:22.111955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0825-e9f0-45e3-b0ef-bf7d670cf294 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:22.112326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0825-e9f0-45e3-b0ef-bf7d670cf294 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:22.112735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0825-e9f0-45e3-b0ef-bf7d670cf294 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:22.120401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b0825-e9f0-45e3-b0ef-bf7d670cf294 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:22.122021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b0825-e9f0-45e3-b0ef-bf7d670cf294 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-ff1b0825-e9f0-45e3-b0ef-bf7d670cf294 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:22.136822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:22.137977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:22.139173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:22.139173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:22.139173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:22.139173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:22.139173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:22.139173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:22.139173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:22.139173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:22.140671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 291/592] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:22.141564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b0825-e9f0-45e3-b0ef-bf7d670cf294 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:22.172924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3f998-616b-418b-a1c7-7f9abaef1735 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:22.218516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06356656-531b-4ed6-bb67-acb67ce1a7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:22.219277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06356656-531b-4ed6-bb67-acb67ce1a7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:22.219564 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06356656-531b-4ed6-bb67-acb67ce1a7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:22.220203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06356656-531b-4ed6-bb67-acb67ce1a7a2 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:22.221757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06356656-531b-4ed6-bb67-acb67ce1a7a2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:22.222082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06356656-531b-4ed6-bb67-acb67ce1a7a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:22.223919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06356656-531b-4ed6-bb67-acb67ce1a7a2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:22.250770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 302/593] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 09:45:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:22.251844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06356656-531b-4ed6-bb67-acb67ce1a7a2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:22.266161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b0825-e9f0-45e3-b0ef-bf7d670cf294 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:22.317285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4678f69-9e7e-4a8d-9cb2-b323f369f631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:22.318589 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4678f69-9e7e-4a8d-9cb2-b323f369f631 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:22.318867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4678f69-9e7e-4a8d-9cb2-b323f369f631 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:22.319400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4678f69-9e7e-4a8d-9cb2-b323f369f631 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:22.331485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4678f69-9e7e-4a8d-9cb2-b323f369f631 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:22.332938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4678f69-9e7e-4a8d-9cb2-b323f369f631 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-b4678f69-9e7e-4a8d-9cb2-b323f369f631 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:22.354516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:22.356767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 292/594] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:22.358086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4678f69-9e7e-4a8d-9cb2-b323f369f631 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:22.365324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06356656-531b-4ed6-bb67-acb67ce1a7a2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:22.375869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:22.395946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:22.409497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96cd33-df64-42a5-90d6-eb2f92a62886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:22.410188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96cd33-df64-42a5-90d6-eb2f92a62886 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:22.410453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96cd33-df64-42a5-90d6-eb2f92a62886 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:22.410755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96cd33-df64-42a5-90d6-eb2f92a62886 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:22.411551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96cd33-df64-42a5-90d6-eb2f92a62886 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:22.411858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96cd33-df64-42a5-90d6-eb2f92a62886 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:22.413310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96cd33-df64-42a5-90d6-eb2f92a62886 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:22.431722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 303/595] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:22.431722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96cd33-df64-42a5-90d6-eb2f92a62886 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:22.517698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4678f69-9e7e-4a8d-9cb2-b323f369f631 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:22.538755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96cd33-df64-42a5-90d6-eb2f92a62886 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:22.567705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c752cb-7871-4480-9b65-35329f88593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:22.569119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c752cb-7871-4480-9b65-35329f88593a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:22.570031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c752cb-7871-4480-9b65-35329f88593a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:22.570748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c752cb-7871-4480-9b65-35329f88593a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:22.574382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad9f4-cbfd-438e-aac0-5c831c5a4cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:22.575208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad9f4-cbfd-438e-aac0-5c831c5a4cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:22.575553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad9f4-cbfd-438e-aac0-5c831c5a4cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:22.575890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad9f4-cbfd-438e-aac0-5c831c5a4cb1 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:22.576896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ad9f4-cbfd-438e-aac0-5c831c5a4cb1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:22.577265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ad9f4-cbfd-438e-aac0-5c831c5a4cb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:22.578951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ad9f4-cbfd-438e-aac0-5c831c5a4cb1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:22.586187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c752cb-7871-4480-9b65-35329f88593a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:22.587748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c752cb-7871-4480-9b65-35329f88593a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:22.602073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 304/596] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Jun 29 09:45:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:22.603335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad9f4-cbfd-438e-aac0-5c831c5a4cb1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-93c752cb-7871-4480-9b65-35329f88593a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:22.609336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:22.610203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:22.611253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:22.611253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:22.611253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:22.611253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:22.611253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 293/597] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:22.611253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c752cb-7871-4480-9b65-35329f88593a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:22.643751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:22.722958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:22.725166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad9f4-cbfd-438e-aac0-5c831c5a4cb1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:22.760201 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c752cb-7871-4480-9b65-35329f88593a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:22.763136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428dc70-5e37-4ab1-9067-e7ee3ed600dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:22.764616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428dc70-5e37-4ab1-9067-e7ee3ed600dd None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:22.764616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428dc70-5e37-4ab1-9067-e7ee3ed600dd None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:22.764893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428dc70-5e37-4ab1-9067-e7ee3ed600dd None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/8e850be24df24bcf972e6e31002014cb/roles/9b6eb8ccaac643cc93781d6abe31bb8f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:22.781369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428dc70-5e37-4ab1-9067-e7ee3ed600dd None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=8e850be24df24bcf972e6e31002014cb, role_id=9b6eb8ccaac643cc93781d6abe31bb8f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:22.783152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428dc70-5e37-4ab1-9067-e7ee3ed600dd None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:22.810613 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 305/598] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:45:22 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/8e850be24df24bcf972e6e31002014cb/roles/9b6eb8ccaac643cc93781d6abe31bb8f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:45:22.810613 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428dc70-5e37-4ab1-9067-e7ee3ed600dd None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:22.814400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f023ab-2c77-45a7-943b-d10c7ffd5b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:22.815915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f023ab-2c77-45a7-943b-d10c7ffd5b92 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:22.815915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f023ab-2c77-45a7-943b-d10c7ffd5b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:22.816118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f023ab-2c77-45a7-943b-d10c7ffd5b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:22.830593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f023ab-2c77-45a7-943b-d10c7ffd5b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:22.831578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f023ab-2c77-45a7-943b-d10c7ffd5b92 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-b5f023ab-2c77-45a7-943b-d10c7ffd5b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:22.854037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:22.855105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:22.856014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:22.856014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:22.856014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:22.856014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:22.856014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 294/599] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:22.860102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f023ab-2c77-45a7-943b-d10c7ffd5b92 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:22.992053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:23.032298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428dc70-5e37-4ab1-9067-e7ee3ed600dd None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:23.070910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f023ab-2c77-45a7-943b-d10c7ffd5b92 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:23.081707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ef6ea-35b4-46f1-a9b4-3c0aea720e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:23.082544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ef6ea-35b4-46f1-a9b4-3c0aea720e8d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:23.082945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ef6ea-35b4-46f1-a9b4-3c0aea720e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:23.083327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ef6ea-35b4-46f1-a9b4-3c0aea720e8d None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:23.084625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ef6ea-35b4-46f1-a9b4-3c0aea720e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:23.084940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ef6ea-35b4-46f1-a9b4-3c0aea720e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:23.086625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ef6ea-35b4-46f1-a9b4-3c0aea720e8d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:23.103243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 306/600] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:23.104844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:23.108380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ef6ea-35b4-46f1-a9b4-3c0aea720e8d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:23.120246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244454d8-d03f-4ad1-a6f1-5486c40f46b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:23.121315 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244454d8-d03f-4ad1-a6f1-5486c40f46b6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:23.121799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244454d8-d03f-4ad1-a6f1-5486c40f46b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:23.122283 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244454d8-d03f-4ad1-a6f1-5486c40f46b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:23.123270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244454d8-d03f-4ad1-a6f1-5486c40f46b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:23.124815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244454d8-d03f-4ad1-a6f1-5486c40f46b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:23.126506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244454d8-d03f-4ad1-a6f1-5486c40f46b6 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:23.141446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 295/601] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:23.148982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244454d8-d03f-4ad1-a6f1-5486c40f46b6 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:23.303187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244454d8-d03f-4ad1-a6f1-5486c40f46b6 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:23.332968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ef6ea-35b4-46f1-a9b4-3c0aea720e8d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:23.348713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03466f21-7b22-4cc8-9091-03f8c933669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:23.350044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03466f21-7b22-4cc8-9091-03f8c933669e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:23.350413 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03466f21-7b22-4cc8-9091-03f8c933669e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:23.350824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03466f21-7b22-4cc8-9091-03f8c933669e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:23.360671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03466f21-7b22-4cc8-9091-03f8c933669e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:23.362972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03466f21-7b22-4cc8-9091-03f8c933669e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:23.377751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c33aa1-5094-4617-ab0b-7d0e318e0bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:23.378982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33aa1-5094-4617-ab0b-7d0e318e0bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:23.379432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33aa1-5094-4617-ab0b-7d0e318e0bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:23.379869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33aa1-5094-4617-ab0b-7d0e318e0bf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-03466f21-7b22-4cc8-9091-03f8c933669e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:23.379970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:23.380786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:23.381672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:23.381672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:23.381672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:23.381672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:23.381672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:23.381672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:23.381672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:23.381672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:23.386630 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 296/602] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:23.388250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b52a36-4944-49be-939b-47c304f831bd None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:23.390204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b52a36-4944-49be-939b-47c304f831bd None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:23.390493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b52a36-4944-49be-939b-47c304f831bd None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:23.392177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 297/603] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:23.393838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b52a36-4944-49be-939b-47c304f831bd None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:23.394446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:23.407986 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c33aa1-5094-4617-ab0b-7d0e318e0bf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:23.411709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c33aa1-5094-4617-ab0b-7d0e318e0bf1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-b4c33aa1-5094-4617-ab0b-7d0e318e0bf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:23.435242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:23.436191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:23.437053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:23.437053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:23.437053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:23.437053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:23.437053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:23.437053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:23.437053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:23.437053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:23.444878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 307/604] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:23.447980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15f44e-207a-44f2-b0f7-a934f7efa283 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:23.448419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15f44e-207a-44f2-b0f7-a934f7efa283 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:23.448834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15f44e-207a-44f2-b0f7-a934f7efa283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:23.451021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:23.475202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0a15f44e-207a-44f2-b0f7-a934f7efa283 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:23.476659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0a15f44e-207a-44f2-b0f7-a934f7efa283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:23.536133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b52a36-4944-49be-939b-47c304f831bd None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:23.566697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 308/605] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:23.566697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15f44e-207a-44f2-b0f7-a934f7efa283 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:23.591522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a058c5-231e-4e41-8f51-654b2e41bc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:23.592203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a058c5-231e-4e41-8f51-654b2e41bc63 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:23.592518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a058c5-231e-4e41-8f51-654b2e41bc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:23.592794 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a058c5-231e-4e41-8f51-654b2e41bc63 None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:23.596774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a058c5-231e-4e41-8f51-654b2e41bc63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:23.596774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a058c5-231e-4e41-8f51-654b2e41bc63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:23.596774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a058c5-231e-4e41-8f51-654b2e41bc63 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:23.620617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 298/606] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:23.620617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a058c5-231e-4e41-8f51-654b2e41bc63 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:23.714864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15f44e-207a-44f2-b0f7-a934f7efa283 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:23.752680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a058c5-231e-4e41-8f51-654b2e41bc63 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:23.754800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:23.761133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:23.770378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eceadf-f5a8-4470-a1bc-3a4815a2fce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:23.771827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eceadf-f5a8-4470-a1bc-3a4815a2fce0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:23.772317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eceadf-f5a8-4470-a1bc-3a4815a2fce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:23.772872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eceadf-f5a8-4470-a1bc-3a4815a2fce0 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:23.774629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eceadf-f5a8-4470-a1bc-3a4815a2fce0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:23.775160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eceadf-f5a8-4470-a1bc-3a4815a2fce0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:23.776951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eceadf-f5a8-4470-a1bc-3a4815a2fce0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:23.798458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 309/607] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:23.798458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eceadf-f5a8-4470-a1bc-3a4815a2fce0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:23.800524 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a5d8-f03f-4a20-8e52-f93d0d58e2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:23.801348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a5d8-f03f-4a20-8e52-f93d0d58e2ff None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:23.801625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a5d8-f03f-4a20-8e52-f93d0d58e2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:23.801943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a5d8-f03f-4a20-8e52-f93d0d58e2ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:23.814772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873a5d8-f03f-4a20-8e52-f93d0d58e2ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-8873a5d8-f03f-4a20-8e52-f93d0d58e2ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:23.827108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:23.828119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 299/608] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:23.829033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a5d8-f03f-4a20-8e52-f93d0d58e2ff None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:23.936693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eceadf-f5a8-4470-a1bc-3a4815a2fce0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:23.950131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a5d8-f03f-4a20-8e52-f93d0d58e2ff None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:23.976693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef12a9-2ece-4f0c-b9d6-ebec21c85869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:23.977597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef12a9-2ece-4f0c-b9d6-ebec21c85869 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:23.977877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef12a9-2ece-4f0c-b9d6-ebec21c85869 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:23.978267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef12a9-2ece-4f0c-b9d6-ebec21c85869 None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/4c97b321bf854177bd2f310cf2958959/roles/9b6eb8ccaac643cc93781d6abe31bb8f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:23.994734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef12a9-2ece-4f0c-b9d6-ebec21c85869 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=4c97b321bf854177bd2f310cf2958959, role_id=9b6eb8ccaac643cc93781d6abe31bb8f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:23.995835 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcd211-460a-4e15-989d-9c7aa76578ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:23.996232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef12a9-2ece-4f0c-b9d6-ebec21c85869 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:23.996764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd211-460a-4e15-989d-9c7aa76578ce None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:23.997047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd211-460a-4e15-989d-9c7aa76578ce None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:23.997321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd211-460a-4e15-989d-9c7aa76578ce None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/34813be7e8fd4fcfb5907bee6a465d16/roles/9b6eb8ccaac643cc93781d6abe31bb8f` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:24.011497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 310/609] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:45:23 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/4c97b321bf854177bd2f310cf2958959/roles/9b6eb8ccaac643cc93781d6abe31bb8f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:45:24.012398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef12a9-2ece-4f0c-b9d6-ebec21c85869 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:24.032903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcd211-460a-4e15-989d-9c7aa76578ce None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=34813be7e8fd4fcfb5907bee6a465d16, role_id=9b6eb8ccaac643cc93781d6abe31bb8f)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:24.035370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcd211-460a-4e15-989d-9c7aa76578ce None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:24.057862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 300/610] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:45:23 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/34813be7e8fd4fcfb5907bee6a465d16/roles/9b6eb8ccaac643cc93781d6abe31bb8f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:45:24.059930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7df624-8ab0-43ce-9897-48a8782991fd None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:24.060576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7df624-8ab0-43ce-9897-48a8782991fd None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:24.061049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7df624-8ab0-43ce-9897-48a8782991fd None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:24.068029 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 301/611] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:24.070990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebbecb-7844-4c25-b4ef-011d48a5f08c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:24.072379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebbecb-7844-4c25-b4ef-011d48a5f08c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:24.074094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebbecb-7844-4c25-b4ef-011d48a5f08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:24.094935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:24.103990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-95ebbecb-7844-4c25-b4ef-011d48a5f08c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:24.106541 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-95ebbecb-7844-4c25-b4ef-011d48a5f08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:24.159041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:24.230819 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 302/612] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:24.270161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebbecb-7844-4c25-b4ef-011d48a5f08c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:24.275508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef12a9-2ece-4f0c-b9d6-ebec21c85869 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:24.317133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c939e1-582c-4bfd-af6e-9fd7802cb575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:24.317832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c939e1-582c-4bfd-af6e-9fd7802cb575 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:24.318115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c939e1-582c-4bfd-af6e-9fd7802cb575 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:24.318393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c939e1-582c-4bfd-af6e-9fd7802cb575 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:24.319240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c939e1-582c-4bfd-af6e-9fd7802cb575 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:24.319552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c939e1-582c-4bfd-af6e-9fd7802cb575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:24.320951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c939e1-582c-4bfd-af6e-9fd7802cb575 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:24.339162 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 311/613] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:24.344553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c939e1-582c-4bfd-af6e-9fd7802cb575 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:24.433888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebbecb-7844-4c25-b4ef-011d48a5f08c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:24.476600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:24.482022 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c939e1-582c-4bfd-af6e-9fd7802cb575 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:24.488038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9841f-efaf-4858-a1c8-f1fe6b01b5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:24.489213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9841f-efaf-4858-a1c8-f1fe6b01b5eb None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:24.489731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9841f-efaf-4858-a1c8-f1fe6b01b5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:24.490862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9841f-efaf-4858-a1c8-f1fe6b01b5eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:24.506391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9841f-efaf-4858-a1c8-f1fe6b01b5eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-65a9841f-efaf-4858-a1c8-f1fe6b01b5eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:24.517934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:24.519049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:24.519937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:24.520616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97daf4e2-1827-4610-beea-bcdb128c1a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:24.520616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daf4e2-1827-4610-beea-bcdb128c1a18 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:24.520616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daf4e2-1827-4610-beea-bcdb128c1a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:24.520616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daf4e2-1827-4610-beea-bcdb128c1a18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:24.523929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 303/614] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:24.535645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9841f-efaf-4858-a1c8-f1fe6b01b5eb None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:24.535858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97daf4e2-1827-4610-beea-bcdb128c1a18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:24.537303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97daf4e2-1827-4610-beea-bcdb128c1a18 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:24.550407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-97daf4e2-1827-4610-beea-bcdb128c1a18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:24.562621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:24.563655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:24.564849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:24.564849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:24.564849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:24.564849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:24.564849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 312/615] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:24.569925 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97daf4e2-1827-4610-beea-bcdb128c1a18 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:24.690046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9841f-efaf-4858-a1c8-f1fe6b01b5eb None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:24.754317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02864a96-5592-45a6-ab2d-a4c51f2debe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:24.755729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02864a96-5592-45a6-ab2d-a4c51f2debe8 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:24.756162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02864a96-5592-45a6-ab2d-a4c51f2debe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:24.756644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02864a96-5592-45a6-ab2d-a4c51f2debe8 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:24.757592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02864a96-5592-45a6-ab2d-a4c51f2debe8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:24.758059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02864a96-5592-45a6-ab2d-a4c51f2debe8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:24.759751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02864a96-5592-45a6-ab2d-a4c51f2debe8 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:24.771163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 304/616] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:24.777082 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02864a96-5592-45a6-ab2d-a4c51f2debe8 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:24.801206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97daf4e2-1827-4610-beea-bcdb128c1a18 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:24.873071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e131c1e-6901-4ad3-8f02-691124c4094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:24.874106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e131c1e-6901-4ad3-8f02-691124c4094a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:24.874400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e131c1e-6901-4ad3-8f02-691124c4094a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:24.874704 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e131c1e-6901-4ad3-8f02-691124c4094a None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:24.875456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e131c1e-6901-4ad3-8f02-691124c4094a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:24.875777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e131c1e-6901-4ad3-8f02-691124c4094a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:24.877836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e131c1e-6901-4ad3-8f02-691124c4094a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:24.902910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:24.905601 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 313/617] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:24.908589 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e131c1e-6901-4ad3-8f02-691124c4094a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:24.934761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:24.945497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02864a96-5592-45a6-ab2d-a4c51f2debe8 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:24.981243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efd7bf-e6d8-439e-a19e-6d6a288b3b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:24.982162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efd7bf-e6d8-439e-a19e-6d6a288b3b38 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:24.982454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efd7bf-e6d8-439e-a19e-6d6a288b3b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:24.982861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efd7bf-e6d8-439e-a19e-6d6a288b3b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:24.994412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efd7bf-e6d8-439e-a19e-6d6a288b3b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:24.995903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efd7bf-e6d8-439e-a19e-6d6a288b3b38 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-46efd7bf-e6d8-439e-a19e-6d6a288b3b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:25.014917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:25.015977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:25.016878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:25.016878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:25.016878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:25.016878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:25.016878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 305/618] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:25.022921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efd7bf-e6d8-439e-a19e-6d6a288b3b38 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:25.140660 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e131c1e-6901-4ad3-8f02-691124c4094a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:25.178199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d9cff-3dbd-401d-9761-373741be3f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:25.179725 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d9cff-3dbd-401d-9761-373741be3f9c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:25.180009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d9cff-3dbd-401d-9761-373741be3f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:25.180288 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d9cff-3dbd-401d-9761-373741be3f9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:25.192498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efd7bf-e6d8-439e-a19e-6d6a288b3b38 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:25.195875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d9cff-3dbd-401d-9761-373741be3f9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:25.197358 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d9cff-3dbd-401d-9761-373741be3f9c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-3e7d9cff-3dbd-401d-9761-373741be3f9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:25.226452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:25.227517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:25.228604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:25.228604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:25.228604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:25.228604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:25.228604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:25.228604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:25.228604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:25.228604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:25.228604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:25.228604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:25.233037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 314/619] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 09:45:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:25.242873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d9cff-3dbd-401d-9761-373741be3f9c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:25.248696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2ba0c-dbd3-4e53-996d-3381330ec15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:25.249381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2ba0c-dbd3-4e53-996d-3381330ec15c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:25.249666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2ba0c-dbd3-4e53-996d-3381330ec15c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:25.249946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2ba0c-dbd3-4e53-996d-3381330ec15c None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:25.250718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2ba0c-dbd3-4e53-996d-3381330ec15c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:25.251043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2ba0c-dbd3-4e53-996d-3381330ec15c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:25.253544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2ba0c-dbd3-4e53-996d-3381330ec15c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:25.257815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:25.278045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 306/620] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:25.288121 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e058e-f211-4b25-a7c8-70cfdf874c9a None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:25.288870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e058e-f211-4b25-a7c8-70cfdf874c9a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:25.289443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e058e-f211-4b25-a7c8-70cfdf874c9a None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:25.292480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 307/621] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:25 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:25.293912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e058e-f211-4b25-a7c8-70cfdf874c9a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:25.335936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:25.370309 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d9cff-3dbd-401d-9761-373741be3f9c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:25.407824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06dec7-c756-410f-a7b9-dbf3f8ad23e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:25.408663 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06dec7-c756-410f-a7b9-dbf3f8ad23e6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:25.408951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06dec7-c756-410f-a7b9-dbf3f8ad23e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:25.409225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06dec7-c756-410f-a7b9-dbf3f8ad23e6 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:25.410073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06dec7-c756-410f-a7b9-dbf3f8ad23e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:25.410385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06dec7-c756-410f-a7b9-dbf3f8ad23e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:25.411897 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06dec7-c756-410f-a7b9-dbf3f8ad23e6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:25.429375 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e058e-f211-4b25-a7c8-70cfdf874c9a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:25.431405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 315/622] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 09:45:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:25.432804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa97bb-8617-43d7-9a1c-d7d5eabc1669 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:25.433094 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa97bb-8617-43d7-9a1c-d7d5eabc1669 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:25.433376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa97bb-8617-43d7-9a1c-d7d5eabc1669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:25.450785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dafa97bb-8617-43d7-9a1c-d7d5eabc1669 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:25.451982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dafa97bb-8617-43d7-9a1c-d7d5eabc1669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:25.473523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d888cfa-e71b-414c-80e6-61755b0c61f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:25.474754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d888cfa-e71b-414c-80e6-61755b0c61f9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:25.475042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d888cfa-e71b-414c-80e6-61755b0c61f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:25.475319 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d888cfa-e71b-414c-80e6-61755b0c61f9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:25.483260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d888cfa-e71b-414c-80e6-61755b0c61f9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:25.484743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d888cfa-e71b-414c-80e6-61755b0c61f9 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-4d888cfa-e71b-414c-80e6-61755b0c61f9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:25.499583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:25.500556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:25.501632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:25.501632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:25.501632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:25.501632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:25.501632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:25.501632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:25.501632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:25.501632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:25.501632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:25.501632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:25.501632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 308/623] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:25.502229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d888cfa-e71b-414c-80e6-61755b0c61f9 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:25.538821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 316/624] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:25.539754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa97bb-8617-43d7-9a1c-d7d5eabc1669 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:25.550847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:25.617074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d888cfa-e71b-414c-80e6-61755b0c61f9 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:25.621326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:25.656856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bc7d0-8ef5-4043-be7a-84df4d2ae300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:25.657889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bc7d0-8ef5-4043-be7a-84df4d2ae300 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:25.658366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bc7d0-8ef5-4043-be7a-84df4d2ae300 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:25.658825 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bc7d0-8ef5-4043-be7a-84df4d2ae300 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:25.666158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa97bb-8617-43d7-9a1c-d7d5eabc1669 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:25.704584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb24e95-1336-42fa-a44e-3696b8ec4d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:25.705270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb24e95-1336-42fa-a44e-3696b8ec4d78 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:25.705555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb24e95-1336-42fa-a44e-3696b8ec4d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:25.705833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb24e95-1336-42fa-a44e-3696b8ec4d78 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:25.706876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb24e95-1336-42fa-a44e-3696b8ec4d78 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:25.707100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb24e95-1336-42fa-a44e-3696b8ec4d78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:25.708567 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb24e95-1336-42fa-a44e-3696b8ec4d78 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:25.728098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 317/625] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 09:45:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:25.729199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb24e95-1336-42fa-a44e-3696b8ec4d78 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:25.736637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bc7d0-8ef5-4043-be7a-84df4d2ae300 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:25.738348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bc7d0-8ef5-4043-be7a-84df4d2ae300 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:25.781228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:25.834708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb24e95-1336-42fa-a44e-3696b8ec4d78 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:25.844135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 309/626] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:25.845500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bc7d0-8ef5-4043-be7a-84df4d2ae300 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:25.853569 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:25.873477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e0655-eba1-40c5-8372-62ecb6abb792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:25.874293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e0655-eba1-40c5-8372-62ecb6abb792 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:25.874598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e0655-eba1-40c5-8372-62ecb6abb792 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:25.874889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e0655-eba1-40c5-8372-62ecb6abb792 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:25.886153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e0655-eba1-40c5-8372-62ecb6abb792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-cf3e0655-eba1-40c5-8372-62ecb6abb792 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:25.894136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:25.895150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:25.895991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:25.900282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 318/627] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:25.901187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e0655-eba1-40c5-8372-62ecb6abb792 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:25.953618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bc7d0-8ef5-4043-be7a-84df4d2ae300 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:25.989772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d7f56-d828-4098-b529-f9f4aa1a6f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:25.991000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d7f56-d828-4098-b529-f9f4aa1a6f61 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:25.991517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d7f56-d828-4098-b529-f9f4aa1a6f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:25.991992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d7f56-d828-4098-b529-f9f4aa1a6f61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.003000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d7f56-d828-4098-b529-f9f4aa1a6f61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:26.004828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d7f56-d828-4098-b529-f9f4aa1a6f61 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:26.008520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e0655-eba1-40c5-8372-62ecb6abb792 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:26.013041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-df6d7f56-d828-4098-b529-f9f4aa1a6f61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:26.026241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:26.027246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:26.028225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:26.028225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:26.028225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:26.028225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:26.030502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 310/628] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:26.031569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d7f56-d828-4098-b529-f9f4aa1a6f61 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:26.046134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d730f19-a4b0-47b1-89b9-b1a8115db9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:26.047004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d730f19-a4b0-47b1-89b9-b1a8115db9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.047336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d730f19-a4b0-47b1-89b9-b1a8115db9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.047623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d730f19-a4b0-47b1-89b9-b1a8115db9e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.060127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d730f19-a4b0-47b1-89b9-b1a8115db9e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:26.061546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d730f19-a4b0-47b1-89b9-b1a8115db9e8 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-2d730f19-a4b0-47b1-89b9-b1a8115db9e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:26.079815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:26.080811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:26.081681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:26.081681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:26.081681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:26.081681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:26.081681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 319/629] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:26.082438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d730f19-a4b0-47b1-89b9-b1a8115db9e8 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:26.113561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:26.146222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d7f56-d828-4098-b529-f9f4aa1a6f61 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:26.183188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d5668-a967-41c8-9249-f516a7f368d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:26.184282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d5668-a967-41c8-9249-f516a7f368d7 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.184765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d5668-a967-41c8-9249-f516a7f368d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.185243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d5668-a967-41c8-9249-f516a7f368d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.186198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d5668-a967-41c8-9249-f516a7f368d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:26.186748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d5668-a967-41c8-9249-f516a7f368d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:26.188364 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d5668-a967-41c8-9249-f516a7f368d7 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:26.190682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d730f19-a4b0-47b1-89b9-b1a8115db9e8 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:26.198462 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 311/630] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:26.199635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d5668-a967-41c8-9249-f516a7f368d7 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:26.233284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96685a5c-22a0-4e50-8703-8d96ff3e5704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:26.233966 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96685a5c-22a0-4e50-8703-8d96ff3e5704 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.234251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96685a5c-22a0-4e50-8703-8d96ff3e5704 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.234533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96685a5c-22a0-4e50-8703-8d96ff3e5704 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.235352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96685a5c-22a0-4e50-8703-8d96ff3e5704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:26.235870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96685a5c-22a0-4e50-8703-8d96ff3e5704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:26.237322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96685a5c-22a0-4e50-8703-8d96ff3e5704 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:26.251004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 320/631] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:26.252146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96685a5c-22a0-4e50-8703-8d96ff3e5704 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:26.262555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:26.311980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d5668-a967-41c8-9249-f516a7f368d7 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:26.345534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:26.349511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a6e8b-dca4-4494-b4bb-90e0feeb1342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:26.350203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a6e8b-dca4-4494-b4bb-90e0feeb1342 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.350482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a6e8b-dca4-4494-b4bb-90e0feeb1342 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.350799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a6e8b-dca4-4494-b4bb-90e0feeb1342 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.351536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a6e8b-dca4-4494-b4bb-90e0feeb1342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:26.351841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a6e8b-dca4-4494-b4bb-90e0feeb1342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:26.353223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a6e8b-dca4-4494-b4bb-90e0feeb1342 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:26.359830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96685a5c-22a0-4e50-8703-8d96ff3e5704 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:26.366545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 312/632] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:26.370893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a6e8b-dca4-4494-b4bb-90e0feeb1342 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:26.398425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bba5c-85c4-41a5-a662-cc56eba28c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:26.399512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bba5c-85c4-41a5-a662-cc56eba28c66 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.399941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bba5c-85c4-41a5-a662-cc56eba28c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.400378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bba5c-85c4-41a5-a662-cc56eba28c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.411420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72bba5c-85c4-41a5-a662-cc56eba28c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:26.413094 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72bba5c-85c4-41a5-a662-cc56eba28c66 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-a72bba5c-85c4-41a5-a662-cc56eba28c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:26.432259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:26.433351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:26.434385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:26.434385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:26.434385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:26.434385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:26.440336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 321/633] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:26.440916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce0163-379f-47a5-9fd3-171cd2f8689a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.441383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce0163-379f-47a5-9fd3-171cd2f8689a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.441383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce0163-379f-47a5-9fd3-171cd2f8689a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.443790 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 322/634] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:26.448717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce0163-379f-47a5-9fd3-171cd2f8689a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:26.474431 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a6e8b-dca4-4494-b4bb-90e0feeb1342 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:26.490345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:26.513155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d9561-afee-42db-966b-b3610b0c7e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:26.513962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9561-afee-42db-966b-b3610b0c7e94 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.514253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9561-afee-42db-966b-b3610b0c7e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.514512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9561-afee-42db-966b-b3610b0c7e94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.521748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d9561-afee-42db-966b-b3610b0c7e94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:26.523237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d9561-afee-42db-966b-b3610b0c7e94 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-a87d9561-afee-42db-966b-b3610b0c7e94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:26.537325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:26.538292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:26.539732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:26.539732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:26.539732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:26.539732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:26.539732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:26.539732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:26.539732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:26.539732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:26.539732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 313/635] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:26.544037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d9561-afee-42db-966b-b3610b0c7e94 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:26.560066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce0163-379f-47a5-9fd3-171cd2f8689a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:26.594308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706af981-e385-41b1-a1cb-3ca7b75a5a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:26.595247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706af981-e385-41b1-a1cb-3ca7b75a5a3c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.595558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706af981-e385-41b1-a1cb-3ca7b75a5a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.595886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706af981-e385-41b1-a1cb-3ca7b75a5a3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.603154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706af981-e385-41b1-a1cb-3ca7b75a5a3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:26.604600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706af981-e385-41b1-a1cb-3ca7b75a5a3c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:26.607215 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-706af981-e385-41b1-a1cb-3ca7b75a5a3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:26.619487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:26.620559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:26.621712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:26.621712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:26.621712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:26.621712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:26.621712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:26.621712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:26.621712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:26.621712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:26.622748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 323/636] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:26.624048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecb4ed-645e-4072-8a8c-255c609b3c4e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.624332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecb4ed-645e-4072-8a8c-255c609b3c4e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.624607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecb4ed-645e-4072-8a8c-255c609b3c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.643041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dfecb4ed-645e-4072-8a8c-255c609b3c4e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:26.644257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dfecb4ed-645e-4072-8a8c-255c609b3c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:26.654352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d9561-afee-42db-966b-b3610b0c7e94 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:26.700477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc1881-3b12-47d7-8ffa-713fedaba728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:26.701154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc1881-3b12-47d7-8ffa-713fedaba728 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.701437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc1881-3b12-47d7-8ffa-713fedaba728 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.701785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc1881-3b12-47d7-8ffa-713fedaba728 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.702438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc1881-3b12-47d7-8ffa-713fedaba728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:26.702774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc1881-3b12-47d7-8ffa-713fedaba728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:26.704151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc1881-3b12-47d7-8ffa-713fedaba728 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:26.716292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 314/637] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:26.717116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc1881-3b12-47d7-8ffa-713fedaba728 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:26.722626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 324/638] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:26.722626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312edf91-94b2-4bcb-b53d-b2638521f280 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.722626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312edf91-94b2-4bcb-b53d-b2638521f280 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.722626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312edf91-94b2-4bcb-b53d-b2638521f280 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.722626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 325/639] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:26.723248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312edf91-94b2-4bcb-b53d-b2638521f280 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:26.759674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:26.827764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc1881-3b12-47d7-8ffa-713fedaba728 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:26.835167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312edf91-94b2-4bcb-b53d-b2638521f280 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:26.862500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcb9f4-0f48-4b0f-afbe-0880982f8612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:26.863205 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcb9f4-0f48-4b0f-afbe-0880982f8612 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.863492 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcb9f4-0f48-4b0f-afbe-0880982f8612 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.863771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcb9f4-0f48-4b0f-afbe-0880982f8612 None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.864570 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcb9f4-0f48-4b0f-afbe-0880982f8612 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:26.864706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:26.864880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcb9f4-0f48-4b0f-afbe-0880982f8612 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:26.866674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcb9f4-0f48-4b0f-afbe-0880982f8612 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:26.871018 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858a933-647b-470b-bee7-64ef0283297b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:26.871622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a933-647b-470b-bee7-64ef0283297b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.871899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a933-647b-470b-bee7-64ef0283297b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.872173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a933-647b-470b-bee7-64ef0283297b None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.872943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858a933-647b-470b-bee7-64ef0283297b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:26.873255 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858a933-647b-470b-bee7-64ef0283297b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:26.874740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858a933-647b-470b-bee7-64ef0283297b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:26.878664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 315/640] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:26.879535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcb9f4-0f48-4b0f-afbe-0880982f8612 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:26.889289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 326/641] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:26.890590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ad647-390a-4055-a452-9cfbcaaa6fcd None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:26.890911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ad647-390a-4055-a452-9cfbcaaa6fcd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:26.891170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ad647-390a-4055-a452-9cfbcaaa6fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:26.908765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ac0ad647-390a-4055-a452-9cfbcaaa6fcd None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:26.909967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ac0ad647-390a-4055-a452-9cfbcaaa6fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:26.977202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 327/642] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:26.978047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ad647-390a-4055-a452-9cfbcaaa6fcd None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:26.986858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcb9f4-0f48-4b0f-afbe-0880982f8612 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:26.994445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:27.030288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedace0-23d6-4c44-a7ca-18a4cf620f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:27.031302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedace0-23d6-4c44-a7ca-18a4cf620f22 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:27.031577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedace0-23d6-4c44-a7ca-18a4cf620f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:27.031851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedace0-23d6-4c44-a7ca-18a4cf620f22 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:27.040466 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedace0-23d6-4c44-a7ca-18a4cf620f22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:27.042258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedace0-23d6-4c44-a7ca-18a4cf620f22 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-9eedace0-23d6-4c44-a7ca-18a4cf620f22 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:27.057629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:27.058864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 316/643] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:27.059843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedace0-23d6-4c44-a7ca-18a4cf620f22 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:27.090071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ad647-390a-4055-a452-9cfbcaaa6fcd None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:27.113227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:27.128891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a983a-1e61-4eb2-921e-8eb3014aa892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:27.129621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a983a-1e61-4eb2-921e-8eb3014aa892 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:27.129902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a983a-1e61-4eb2-921e-8eb3014aa892 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:27.130179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a983a-1e61-4eb2-921e-8eb3014aa892 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:27.140711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a983a-1e61-4eb2-921e-8eb3014aa892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-662a983a-1e61-4eb2-921e-8eb3014aa892 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:27.148819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:27.149768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:27.150696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 328/644] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:27.151511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a983a-1e61-4eb2-921e-8eb3014aa892 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:27.179455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedace0-23d6-4c44-a7ca-18a4cf620f22 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:27.214608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71de526-fed5-427d-9d61-83082eaf3480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:27.215510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71de526-fed5-427d-9d61-83082eaf3480 None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:27.215809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71de526-fed5-427d-9d61-83082eaf3480 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:27.216084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71de526-fed5-427d-9d61-83082eaf3480 None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/d86d073ef5ee4631b508ab578662be96/roles/9b6eb8ccaac643cc93781d6abe31bb8f` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:27.232403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71de526-fed5-427d-9d61-83082eaf3480 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=d86d073ef5ee4631b508ab578662be96, role_id=9b6eb8ccaac643cc93781d6abe31bb8f)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:27.233856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71de526-fed5-427d-9d61-83082eaf3480 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:27.247036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 317/645] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:45:27 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/d86d073ef5ee4631b508ab578662be96/roles/9b6eb8ccaac643cc93781d6abe31bb8f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:45:27.251446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a983a-1e61-4eb2-921e-8eb3014aa892 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:27.251643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71de526-fed5-427d-9d61-83082eaf3480 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:27.260761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:27.292940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6254814-a7a9-43f7-923d-93ee4dc15dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:27.293727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6254814-a7a9-43f7-923d-93ee4dc15dec None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:27.293985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6254814-a7a9-43f7-923d-93ee4dc15dec None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:27.294256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6254814-a7a9-43f7-923d-93ee4dc15dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:27.295270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6254814-a7a9-43f7-923d-93ee4dc15dec None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=8a5eea7f539048c48ed10e72dd933df9) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:27.295585 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6254814-a7a9-43f7-923d-93ee4dc15dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:27.297113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6254814-a7a9-43f7-923d-93ee4dc15dec None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:27.314596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 329/646] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:27 2022] GET /identity/v3/role_assignments?user.id=8a5eea7f539048c48ed10e72dd933df9&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:27.314596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6254814-a7a9-43f7-923d-93ee4dc15dec None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:27.384957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:27.409253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71de526-fed5-427d-9d61-83082eaf3480 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:27.461004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907e319-7646-412c-ba71-bb1ad9c91b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:27.462153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907e319-7646-412c-ba71-bb1ad9c91b1a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:27.462378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6254814-a7a9-43f7-923d-93ee4dc15dec None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:27.462540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907e319-7646-412c-ba71-bb1ad9c91b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:27.463007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907e319-7646-412c-ba71-bb1ad9c91b1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:27.482161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907e319-7646-412c-ba71-bb1ad9c91b1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-8907e319-7646-412c-ba71-bb1ad9c91b1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:27.492742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:27.493953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:27.495001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 318/647] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:27.495871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907e319-7646-412c-ba71-bb1ad9c91b1a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:27.513780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11724413-c1d9-47f8-a6f3-2a20bd5a1a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:27.514473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11724413-c1d9-47f8-a6f3-2a20bd5a1a3f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:27.514838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11724413-c1d9-47f8-a6f3-2a20bd5a1a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:27.515218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11724413-c1d9-47f8-a6f3-2a20bd5a1a3f None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:27.516179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11724413-c1d9-47f8-a6f3-2a20bd5a1a3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:27.516490 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11724413-c1d9-47f8-a6f3-2a20bd5a1a3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:27.519015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11724413-c1d9-47f8-a6f3-2a20bd5a1a3f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:27.540895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 330/648] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 09:45:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:27.541887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11724413-c1d9-47f8-a6f3-2a20bd5a1a3f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:27.574926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:27.627937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907e319-7646-412c-ba71-bb1ad9c91b1a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:27.669683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11724413-c1d9-47f8-a6f3-2a20bd5a1a3f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:27.671322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:27.674583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a4449-84f7-4164-bc5e-bf212258e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:27.675551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a4449-84f7-4164-bc5e-bf212258e11b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:27.675872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a4449-84f7-4164-bc5e-bf212258e11b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:27.676149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a4449-84f7-4164-bc5e-bf212258e11b None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:27.676920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a4449-84f7-4164-bc5e-bf212258e11b None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:27.677234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a4449-84f7-4164-bc5e-bf212258e11b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:27.678667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a4449-84f7-4164-bc5e-bf212258e11b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:27.701651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 319/649] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:27.707296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a4449-84f7-4164-bc5e-bf212258e11b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:27.723007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfadac-e7d9-4768-9cbc-a08312526986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:27.723750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfadac-e7d9-4768-9cbc-a08312526986 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:27.724386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfadac-e7d9-4768-9cbc-a08312526986 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:27.724672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfadac-e7d9-4768-9cbc-a08312526986 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:27.725465 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfadac-e7d9-4768-9cbc-a08312526986 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:27.725770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfadac-e7d9-4768-9cbc-a08312526986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:27.727240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfadac-e7d9-4768-9cbc-a08312526986 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:27.736919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 331/650] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:27.737771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfadac-e7d9-4768-9cbc-a08312526986 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:27.850252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a4449-84f7-4164-bc5e-bf212258e11b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:27.865140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfadac-e7d9-4768-9cbc-a08312526986 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:27.884173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:27.895887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded59e60-37ba-46e5-8c0c-cd9eebf172ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:27.896557 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59e60-37ba-46e5-8c0c-cd9eebf172ec None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:27.896840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59e60-37ba-46e5-8c0c-cd9eebf172ec None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:27.897119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59e60-37ba-46e5-8c0c-cd9eebf172ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:27.903278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6af928-d621-4e46-8aaa-62dd8147a5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:27.904286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6af928-d621-4e46-8aaa-62dd8147a5ba None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:27.904734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6af928-d621-4e46-8aaa-62dd8147a5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:27.905141 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6af928-d621-4e46-8aaa-62dd8147a5ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:27.917453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6af928-d621-4e46-8aaa-62dd8147a5ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:27.919284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6af928-d621-4e46-8aaa-62dd8147a5ba None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-ef6af928-d621-4e46-8aaa-62dd8147a5ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:27.937158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:27.938289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:27.939520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:27.939520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:27.939520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:27.939520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:27.945780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 332/651] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:27.946877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6af928-d621-4e46-8aaa-62dd8147a5ba None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:27.966890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:27.979111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded59e60-37ba-46e5-8c0c-cd9eebf172ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:27.981055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded59e60-37ba-46e5-8c0c-cd9eebf172ec None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:28.074865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6af928-d621-4e46-8aaa-62dd8147a5ba None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:28.107340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 320/652] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:28.108619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded59e60-37ba-46e5-8c0c-cd9eebf172ec None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:28.119673 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fc720-5773-4928-a442-3190602a97ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:28.120914 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fc720-5773-4928-a442-3190602a97ff None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:28.121218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fc720-5773-4928-a442-3190602a97ff None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:28.121493 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fc720-5773-4928-a442-3190602a97ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:28.136221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fc720-5773-4928-a442-3190602a97ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:28.138421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fc720-5773-4928-a442-3190602a97ff None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:28.139555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-7b7fc720-5773-4928-a442-3190602a97ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:28.158170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:28.159328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:28.160296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:28.160296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:28.160296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:28.160296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:28.162213 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 333/653] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:28.163174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fc720-5773-4928-a442-3190602a97ff None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:28.235004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:28.250577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded59e60-37ba-46e5-8c0c-cd9eebf172ec None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:28.272548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fc720-5773-4928-a442-3190602a97ff None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:28.296538 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479a20e-fd84-4952-bebd-71f81b2917a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:28.297854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479a20e-fd84-4952-bebd-71f81b2917a9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:28.298330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479a20e-fd84-4952-bebd-71f81b2917a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:28.299027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479a20e-fd84-4952-bebd-71f81b2917a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:28.300144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479a20e-fd84-4952-bebd-71f81b2917a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:28.300595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479a20e-fd84-4952-bebd-71f81b2917a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:28.302810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479a20e-fd84-4952-bebd-71f81b2917a9 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:28.309084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476fdd0-1502-491f-a8f1-2a60f36a4a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:28.309998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476fdd0-1502-491f-a8f1-2a60f36a4a6c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:28.310370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476fdd0-1502-491f-a8f1-2a60f36a4a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:28.310804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476fdd0-1502-491f-a8f1-2a60f36a4a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:28.317421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 321/654] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:28.318558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479a20e-fd84-4952-bebd-71f81b2917a9 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:28.327509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476fdd0-1502-491f-a8f1-2a60f36a4a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:28.329101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476fdd0-1502-491f-a8f1-2a60f36a4a6c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-2476fdd0-1502-491f-a8f1-2a60f36a4a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:28.347614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:28.348712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:28.350207 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:28.350207 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:28.350207 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:28.350207 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:28.356005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 334/655] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:28.356999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476fdd0-1502-491f-a8f1-2a60f36a4a6c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:28.420426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:28.442339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479a20e-fd84-4952-bebd-71f81b2917a9 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:28.479076 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476fdd0-1502-491f-a8f1-2a60f36a4a6c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:28.491013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0320bc7-ff6a-4073-a95f-0ac992db4ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:28.492147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0320bc7-ff6a-4073-a95f-0ac992db4ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:28.492574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0320bc7-ff6a-4073-a95f-0ac992db4ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:28.493250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0320bc7-ff6a-4073-a95f-0ac992db4ee4 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:28.494687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0320bc7-ff6a-4073-a95f-0ac992db4ee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:28.495411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0320bc7-ff6a-4073-a95f-0ac992db4ee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:28.497554 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0320bc7-ff6a-4073-a95f-0ac992db4ee4 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:28.500671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:28.520521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8597251-959b-4c00-b03b-958c8d198963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:28.520521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597251-959b-4c00-b03b-958c8d198963 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:28.520521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597251-959b-4c00-b03b-958c8d198963 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:28.520521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597251-959b-4c00-b03b-958c8d198963 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:28.521578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 322/656] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:28.523397 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0320bc7-ff6a-4073-a95f-0ac992db4ee4 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:28.528821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8597251-959b-4c00-b03b-958c8d198963 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:28.530305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8597251-959b-4c00-b03b-958c8d198963 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-d8597251-959b-4c00-b03b-958c8d198963 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:28.545806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:28.546887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:28.547812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:28.547812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:28.547812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:28.547812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:28.547812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:28.547812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:28.547812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:28.547812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:28.547812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 09:45:28.547812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:28.547812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 335/657] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:28.548502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8597251-959b-4c00-b03b-958c8d198963 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:28.648980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0320bc7-ff6a-4073-a95f-0ac992db4ee4 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:28.658317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8597251-959b-4c00-b03b-958c8d198963 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:28.686657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36259ee1-e220-4a34-8188-29678d18fbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:28.687734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36259ee1-e220-4a34-8188-29678d18fbaf None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:28.688210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36259ee1-e220-4a34-8188-29678d18fbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:28.688938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36259ee1-e220-4a34-8188-29678d18fbaf None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:28.690158 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36259ee1-e220-4a34-8188-29678d18fbaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:28.690692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36259ee1-e220-4a34-8188-29678d18fbaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:28.692383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36259ee1-e220-4a34-8188-29678d18fbaf None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:28.696323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c99a0d-fdb3-4d32-8c2f-53563c7aa08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:28.697135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c99a0d-fdb3-4d32-8c2f-53563c7aa08f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:28.697409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c99a0d-fdb3-4d32-8c2f-53563c7aa08f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:28.697673 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c99a0d-fdb3-4d32-8c2f-53563c7aa08f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:28.707531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c99a0d-fdb3-4d32-8c2f-53563c7aa08f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:28.707884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 323/658] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:28.708980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36259ee1-e220-4a34-8188-29678d18fbaf None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:28.709089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c99a0d-fdb3-4d32-8c2f-53563c7aa08f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:28.720007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-c4c99a0d-fdb3-4d32-8c2f-53563c7aa08f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:28.729584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:28.730629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:28.732393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:28.732393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:28.732393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:28.732393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:28.732393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:28.732393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:28.732393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:28.732393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:28.732393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 09:45:28.732393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:28.741296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 336/659] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:28.742739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c99a0d-fdb3-4d32-8c2f-53563c7aa08f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:28.789644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:28.825207 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36259ee1-e220-4a34-8188-29678d18fbaf None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:28.854503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c99a0d-fdb3-4d32-8c2f-53563c7aa08f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:28.862449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea391244-8789-4a12-ad6a-d75c158d8098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:28.863354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea391244-8789-4a12-ad6a-d75c158d8098 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:28.863702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea391244-8789-4a12-ad6a-d75c158d8098 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:28.864046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea391244-8789-4a12-ad6a-d75c158d8098 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:28.864960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea391244-8789-4a12-ad6a-d75c158d8098 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:28.865333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea391244-8789-4a12-ad6a-d75c158d8098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:28.867008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea391244-8789-4a12-ad6a-d75c158d8098 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:28.891522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 324/660] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Jun 29 09:45:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:28.891522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea391244-8789-4a12-ad6a-d75c158d8098 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:28.897003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55124f82-a22a-46da-8e55-861cf813243b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:28.898455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55124f82-a22a-46da-8e55-861cf813243b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:28.898974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55124f82-a22a-46da-8e55-861cf813243b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:28.899538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55124f82-a22a-46da-8e55-861cf813243b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:28.914605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55124f82-a22a-46da-8e55-861cf813243b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:28.914605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55124f82-a22a-46da-8e55-861cf813243b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-55124f82-a22a-46da-8e55-861cf813243b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:28.929555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:28.930517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:28.931514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:28.931514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:28.931514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:28.931514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:28.931514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:28.931514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:28.931514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:28.931514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:28.931514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 337/661] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:28.932521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55124f82-a22a-46da-8e55-861cf813243b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:28.958970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:29.002942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea391244-8789-4a12-ad6a-d75c158d8098 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:29.045454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd736d2-8b64-4cc4-8d0e-c61a6dc277ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:29.046436 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd736d2-8b64-4cc4-8d0e-c61a6dc277ea None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:29.046758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd736d2-8b64-4cc4-8d0e-c61a6dc277ea None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:29.047055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd736d2-8b64-4cc4-8d0e-c61a6dc277ea None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:29.049030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd736d2-8b64-4cc4-8d0e-c61a6dc277ea None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:29.050015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd736d2-8b64-4cc4-8d0e-c61a6dc277ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:29.051333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd736d2-8b64-4cc4-8d0e-c61a6dc277ea None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:29.053785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:29.058457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55124f82-a22a-46da-8e55-861cf813243b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:29.076601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 325/662] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 09:45:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:29.078024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd736d2-8b64-4cc4-8d0e-c61a6dc277ea None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:29.102448 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f761f-5c91-45f9-9cc6-ccce0a7b6d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:29.103689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f761f-5c91-45f9-9cc6-ccce0a7b6d3f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:29.104224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f761f-5c91-45f9-9cc6-ccce0a7b6d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:29.104701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f761f-5c91-45f9-9cc6-ccce0a7b6d3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:29.191862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f761f-5c91-45f9-9cc6-ccce0a7b6d3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:29.193773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f761f-5c91-45f9-9cc6-ccce0a7b6d3f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:29.210050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd736d2-8b64-4cc4-8d0e-c61a6dc277ea None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:29.243106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:29.258383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd1eed-2a60-44b0-8870-3e86805bc2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:29.259009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1eed-2a60-44b0-8870-3e86805bc2ba None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:29.259292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1eed-2a60-44b0-8870-3e86805bc2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:29.259566 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1eed-2a60-44b0-8870-3e86805bc2ba None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:29.260639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd1eed-2a60-44b0-8870-3e86805bc2ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:29.260955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd1eed-2a60-44b0-8870-3e86805bc2ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:29.263161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd1eed-2a60-44b0-8870-3e86805bc2ba None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:29.278739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 326/663] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:29.281177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd1eed-2a60-44b0-8870-3e86805bc2ba None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:29.316929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:29.322240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 338/664] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:29.323774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f761f-5c91-45f9-9cc6-ccce0a7b6d3f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:29.421975 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd1eed-2a60-44b0-8870-3e86805bc2ba None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:29.456377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f761f-5c91-45f9-9cc6-ccce0a7b6d3f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:29.467137 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7acdd3-0b62-400a-97f6-15062c97b986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:29.467137 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7acdd3-0b62-400a-97f6-15062c97b986 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:29.467409 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7acdd3-0b62-400a-97f6-15062c97b986 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:29.467666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7acdd3-0b62-400a-97f6-15062c97b986 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:29.500381 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5ed21-ce4b-4619-abd2-107d41036a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:29.501081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5ed21-ce4b-4619-abd2-107d41036a7a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:29.501417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5ed21-ce4b-4619-abd2-107d41036a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:29.502364 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5ed21-ce4b-4619-abd2-107d41036a7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:29.503282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5ed21-ce4b-4619-abd2-107d41036a7a None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=8e850be24df24bcf972e6e31002014cb) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:29.503600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5ed21-ce4b-4619-abd2-107d41036a7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:29.505054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5ed21-ce4b-4619-abd2-107d41036a7a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:29.511483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 339/665] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:29 2022] GET /identity/v3/role_assignments?user.id=8e850be24df24bcf972e6e31002014cb&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:29.512238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5ed21-ce4b-4619-abd2-107d41036a7a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:29.541513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:29.572370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7acdd3-0b62-400a-97f6-15062c97b986 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:29.573931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7acdd3-0b62-400a-97f6-15062c97b986 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:29.608039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:29.662969 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5ed21-ce4b-4619-abd2-107d41036a7a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:29.706864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fd496-7a3a-424f-a04f-f82a5e427597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:29.707680 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fd496-7a3a-424f-a04f-f82a5e427597 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:29.707955 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fd496-7a3a-424f-a04f-f82a5e427597 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:29.708226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fd496-7a3a-424f-a04f-f82a5e427597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:29.712926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 327/666] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:29.714225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255eb471-0e0c-44cf-bc2f-02bfba1ffb28 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:29.714518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255eb471-0e0c-44cf-bc2f-02bfba1ffb28 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:29.714872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255eb471-0e0c-44cf-bc2f-02bfba1ffb28 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:29.716376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 328/667] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:29.717634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712116bd-5641-4f1a-beb2-4389e1ab1402 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:29.718026 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712116bd-5641-4f1a-beb2-4389e1ab1402 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:29.718274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712116bd-5641-4f1a-beb2-4389e1ab1402 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:29.719833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 329/668] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:29.722541 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe5e67-f42c-4444-92e8-9bb138eac628 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:29.723104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe5e67-f42c-4444-92e8-9bb138eac628 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:29.724596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe5e67-f42c-4444-92e8-9bb138eac628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:29.731285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fd496-7a3a-424f-a04f-f82a5e427597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:29.732977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fd496-7a3a-424f-a04f-f82a5e427597 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:29.744930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-17fe5e67-f42c-4444-92e8-9bb138eac628 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:29.746131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-17fe5e67-f42c-4444-92e8-9bb138eac628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f85fd496-7a3a-424f-a04f-f82a5e427597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:29.754696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:29.755645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:29.756531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:29.756531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:29.756531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:29.756531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:29.756531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 340/669] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:29.757170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c3ead-5380-42f3-a225-9101ab252622 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:29.757456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c3ead-5380-42f3-a225-9101ab252622 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:29.757730 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c3ead-5380-42f3-a225-9101ab252622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:29.775871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7d9c3ead-5380-42f3-a225-9101ab252622 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:29.777001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7d9c3ead-5380-42f3-a225-9101ab252622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:29.837152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 330/670] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 09:45:29.846730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe5e67-f42c-4444-92e8-9bb138eac628 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:29.855242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:29.881237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 341/671] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:29.882128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c3ead-5380-42f3-a225-9101ab252622 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:29.935518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:29.973286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe5e67-f42c-4444-92e8-9bb138eac628 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:30.001284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c3ead-5380-42f3-a225-9101ab252622 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:30.014290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54765858-e9ea-46da-893f-5a3357f82012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:30.015266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54765858-e9ea-46da-893f-5a3357f82012 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:30.015560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54765858-e9ea-46da-893f-5a3357f82012 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:30.015940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54765858-e9ea-46da-893f-5a3357f82012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:30.028705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54765858-e9ea-46da-893f-5a3357f82012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:30.030899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54765858-e9ea-46da-893f-5a3357f82012 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:30.037731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb6b45-ab71-4a9e-91c3-44b6dd961116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:30.038418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb6b45-ab71-4a9e-91c3-44b6dd961116 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:30.038740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb6b45-ab71-4a9e-91c3-44b6dd961116 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:30.039016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb6b45-ab71-4a9e-91c3-44b6dd961116 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:30.039760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb6b45-ab71-4a9e-91c3-44b6dd961116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:30.040080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb6b45-ab71-4a9e-91c3-44b6dd961116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:30.041489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb6b45-ab71-4a9e-91c3-44b6dd961116 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-54765858-e9ea-46da-893f-5a3357f82012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:30.053906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:30.054851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:30.060808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:30.060808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:30.060808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:30.060808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:30.060808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 331/672] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:30.060808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54765858-e9ea-46da-893f-5a3357f82012 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:30.062436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 342/673] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:30.063410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb6b45-ab71-4a9e-91c3-44b6dd961116 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:30.140152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:30.192380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb6b45-ab71-4a9e-91c3-44b6dd961116 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:30.192850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54765858-e9ea-46da-893f-5a3357f82012 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:30.202201 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:30.235428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f591a-fee4-4e7d-9506-07c5909691ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:30.236401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f591a-fee4-4e7d-9506-07c5909691ed None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:30.236681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f591a-fee4-4e7d-9506-07c5909691ed None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:30.236953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f591a-fee4-4e7d-9506-07c5909691ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:30.237193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b298a31-deec-4b16-8c6e-0c2f1023046b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:30.238062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b298a31-deec-4b16-8c6e-0c2f1023046b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:30.238404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b298a31-deec-4b16-8c6e-0c2f1023046b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:30.238762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b298a31-deec-4b16-8c6e-0c2f1023046b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:30.251984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f591a-fee4-4e7d-9506-07c5909691ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:30.255542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b298a31-deec-4b16-8c6e-0c2f1023046b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-e98f591a-fee4-4e7d-9506-07c5909691ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:30.258887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:30.260182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:30.261145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 343/674] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:30.262452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f591a-fee4-4e7d-9506-07c5909691ed None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-0b298a31-deec-4b16-8c6e-0c2f1023046b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:30.271814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:30.272810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:30.273643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 332/675] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:30.274322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b298a31-deec-4b16-8c6e-0c2f1023046b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:30.393820 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f591a-fee4-4e7d-9506-07c5909691ed None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:30.397529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b298a31-deec-4b16-8c6e-0c2f1023046b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:30.424048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:30.433930 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9baf51-cdcb-4fdf-bd56-3eee2bbd990e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:30.434657 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9baf51-cdcb-4fdf-bd56-3eee2bbd990e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:30.434916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9baf51-cdcb-4fdf-bd56-3eee2bbd990e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:30.435187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9baf51-cdcb-4fdf-bd56-3eee2bbd990e None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:30.435773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639af0b-62da-4b76-a9d7-82857c19ab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:30.435945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9baf51-cdcb-4fdf-bd56-3eee2bbd990e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:30.436214 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9baf51-cdcb-4fdf-bd56-3eee2bbd990e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:30.436579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639af0b-62da-4b76-a9d7-82857c19ab10 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:30.436859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639af0b-62da-4b76-a9d7-82857c19ab10 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:30.437130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639af0b-62da-4b76-a9d7-82857c19ab10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:30.437675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9baf51-cdcb-4fdf-bd56-3eee2bbd990e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:30.445544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639af0b-62da-4b76-a9d7-82857c19ab10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:30.447043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639af0b-62da-4b76-a9d7-82857c19ab10 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:30.458958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 344/676] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:30.458958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9baf51-cdcb-4fdf-bd56-3eee2bbd990e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-f639af0b-62da-4b76-a9d7-82857c19ab10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:30.468116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:30.469091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:30.470482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:30.470482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:30.470482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:30.470482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:30.470482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:30.470482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:30.470482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:30.470482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:30.474505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 333/677] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:30.475538 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639af0b-62da-4b76-a9d7-82857c19ab10 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:30.495148 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:30.597003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9baf51-cdcb-4fdf-bd56-3eee2bbd990e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:30.603576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639af0b-62da-4b76-a9d7-82857c19ab10 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:30.639068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a27db8-4494-4792-9df3-00a06a7c73dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:30.639768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a27db8-4494-4792-9df3-00a06a7c73dc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:30.640039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a27db8-4494-4792-9df3-00a06a7c73dc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:30.640314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a27db8-4494-4792-9df3-00a06a7c73dc None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:30.641073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a27db8-4494-4792-9df3-00a06a7c73dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:30.641394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a27db8-4494-4792-9df3-00a06a7c73dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:30.643181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9dace-2355-4786-8eb7-27b3d2900fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:30.643330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a27db8-4494-4792-9df3-00a06a7c73dc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:30.644252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9dace-2355-4786-8eb7-27b3d2900fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:30.644727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9dace-2355-4786-8eb7-27b3d2900fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:30.645001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9dace-2355-4786-8eb7-27b3d2900fe1 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:30.646037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9dace-2355-4786-8eb7-27b3d2900fe1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:30.646171 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9dace-2355-4786-8eb7-27b3d2900fe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:30.647461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9dace-2355-4786-8eb7-27b3d2900fe1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:30.653682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 345/678] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:30.658640 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a27db8-4494-4792-9df3-00a06a7c73dc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:30.669299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 334/679] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:30.669299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9dace-2355-4786-8eb7-27b3d2900fe1 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:30.720379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:30.773396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:30.789584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a27db8-4494-4792-9df3-00a06a7c73dc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:30.793293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9dace-2355-4786-8eb7-27b3d2900fe1 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:30.836053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048a1b4-1c98-4b83-af95-f40f5c2f65c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:30.836730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048a1b4-1c98-4b83-af95-f40f5c2f65c6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:30.837009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048a1b4-1c98-4b83-af95-f40f5c2f65c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:30.837284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048a1b4-1c98-4b83-af95-f40f5c2f65c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:30.838082 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048a1b4-1c98-4b83-af95-f40f5c2f65c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:30.838392 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048a1b4-1c98-4b83-af95-f40f5c2f65c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:30.838855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bce4a-1ddc-4705-8383-f56c7c8c2e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:30.839731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bce4a-1ddc-4705-8383-f56c7c8c2e97 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:30.839884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048a1b4-1c98-4b83-af95-f40f5c2f65c6 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:30.840116 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bce4a-1ddc-4705-8383-f56c7c8c2e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:30.840446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bce4a-1ddc-4705-8383-f56c7c8c2e97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:30.848828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5bce4a-1ddc-4705-8383-f56c7c8c2e97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:30.850771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5bce4a-1ddc-4705-8383-f56c7c8c2e97 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:30.861640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 335/680] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:30.861640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048a1b4-1c98-4b83-af95-f40f5c2f65c6 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-da5bce4a-1ddc-4705-8383-f56c7c8c2e97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:30.876390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:30.877356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:30.878307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:30.878307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:30.878307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:30.878307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:30.878307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:30.878307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:30.878307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:30.878307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:30.878778 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 346/681] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:30.880016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bce4a-1ddc-4705-8383-f56c7c8c2e97 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:31.013622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048a1b4-1c98-4b83-af95-f40f5c2f65c6 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:31.023128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bce4a-1ddc-4705-8383-f56c7c8c2e97 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:31.023663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:31.067826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8f0bb-1cfa-464d-92ef-ddd373a9aa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:31.068664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8f0bb-1cfa-464d-92ef-ddd373a9aa8c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:31.068934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8f0bb-1cfa-464d-92ef-ddd373a9aa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:31.069261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8f0bb-1cfa-464d-92ef-ddd373a9aa8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:31.072882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a5860-6366-4cb9-91da-13c571fa225e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:31.074042 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a5860-6366-4cb9-91da-13c571fa225e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:31.074042 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a5860-6366-4cb9-91da-13c571fa225e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:31.074042 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a5860-6366-4cb9-91da-13c571fa225e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:31.083547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8f0bb-1cfa-464d-92ef-ddd373a9aa8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:31.085118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8f0bb-1cfa-464d-92ef-ddd373a9aa8c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:31.088285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a5860-6366-4cb9-91da-13c571fa225e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:31.089732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a5860-6366-4cb9-91da-13c571fa225e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:31.100783 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-5fd8f0bb-1cfa-464d-92ef-ddd373a9aa8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:31.107644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:31.108535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:31.109403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:31.109403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:31.109403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:31.109403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:31.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 336/682] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:31.112774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8f0bb-1cfa-464d-92ef-ddd373a9aa8c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-340a5860-6366-4cb9-91da-13c571fa225e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:31.114708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:31.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:31.116731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:31.116731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:31.116731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:31.116731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:31.116731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 347/683] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:31.117028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a5860-6366-4cb9-91da-13c571fa225e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:31.236097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a5860-6366-4cb9-91da-13c571fa225e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:31.258652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8f0bb-1cfa-464d-92ef-ddd373a9aa8c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:31.278944 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f195d4b-1795-4839-b43e-cd1614eeca0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:31.279535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f195d4b-1795-4839-b43e-cd1614eeca0a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:31.279822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f195d4b-1795-4839-b43e-cd1614eeca0a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:31.280072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f195d4b-1795-4839-b43e-cd1614eeca0a None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:31.280870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f195d4b-1795-4839-b43e-cd1614eeca0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:31.281187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f195d4b-1795-4839-b43e-cd1614eeca0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:31.282953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f195d4b-1795-4839-b43e-cd1614eeca0a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:31.297449 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 348/684] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:31.298010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f195d4b-1795-4839-b43e-cd1614eeca0a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:31.306281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5353785-bb55-47f0-9f67-77e2ad988fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:31.307006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5353785-bb55-47f0-9f67-77e2ad988fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:31.307282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5353785-bb55-47f0-9f67-77e2ad988fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:31.307554 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5353785-bb55-47f0-9f67-77e2ad988fb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:31.308390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5353785-bb55-47f0-9f67-77e2ad988fb5 None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=4c97b321bf854177bd2f310cf2958959) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:31.308697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5353785-bb55-47f0-9f67-77e2ad988fb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:31.310154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5353785-bb55-47f0-9f67-77e2ad988fb5 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:31.318625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 337/685] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:31 2022] GET /identity/v3/role_assignments?user.id=4c97b321bf854177bd2f310cf2958959&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:31.323617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5353785-bb55-47f0-9f67-77e2ad988fb5 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:31.354550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:31.397369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:31.459884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f195d4b-1795-4839-b43e-cd1614eeca0a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:31.460731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5353785-bb55-47f0-9f67-77e2ad988fb5 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:31.510821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b0e69-64fd-4edb-ba7f-3684fb7357c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:31.511520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b0e69-64fd-4edb-ba7f-3684fb7357c9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:31.511803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b0e69-64fd-4edb-ba7f-3684fb7357c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:31.512134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b0e69-64fd-4edb-ba7f-3684fb7357c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:31.513428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b0e69-64fd-4edb-ba7f-3684fb7357c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:31.513428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b0e69-64fd-4edb-ba7f-3684fb7357c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:31.514736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b0e69-64fd-4edb-ba7f-3684fb7357c9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:31.523504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32917c-e6c0-409f-946a-fd06220781e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:31.524297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32917c-e6c0-409f-946a-fd06220781e3 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:31.524579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32917c-e6c0-409f-946a-fd06220781e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:31.524884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32917c-e6c0-409f-946a-fd06220781e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:31.525599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32917c-e6c0-409f-946a-fd06220781e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:31.525962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32917c-e6c0-409f-946a-fd06220781e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:31.528820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32917c-e6c0-409f-946a-fd06220781e3 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:31.533386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 349/686] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:31.534188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b0e69-64fd-4edb-ba7f-3684fb7357c9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:31.542389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 338/687] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:31.544063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32917c-e6c0-409f-946a-fd06220781e3 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:31.668232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b0e69-64fd-4edb-ba7f-3684fb7357c9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:31.677423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:31.698274 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:31.710293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32917c-e6c0-409f-946a-fd06220781e3 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:31.719561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fb1ed-8180-4612-aac6-e67069d814a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:31.720365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fb1ed-8180-4612-aac6-e67069d814a3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:31.720655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fb1ed-8180-4612-aac6-e67069d814a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:31.720954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fb1ed-8180-4612-aac6-e67069d814a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:31.721948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fb1ed-8180-4612-aac6-e67069d814a3 None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=34813be7e8fd4fcfb5907bee6a465d16) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:31.722312 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fb1ed-8180-4612-aac6-e67069d814a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:31.723818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fb1ed-8180-4612-aac6-e67069d814a3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:31.731070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 350/688] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:31 2022] GET /identity/v3/role_assignments?user.id=34813be7e8fd4fcfb5907bee6a465d16&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:31.738617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fb1ed-8180-4612-aac6-e67069d814a3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:31.766755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215205d-ef8c-4bd1-9660-cf4cf37641f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:31.768691 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215205d-ef8c-4bd1-9660-cf4cf37641f0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:31.769137 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215205d-ef8c-4bd1-9660-cf4cf37641f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:31.769554 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215205d-ef8c-4bd1-9660-cf4cf37641f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:31.780420 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8215205d-ef8c-4bd1-9660-cf4cf37641f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:31.783066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8215205d-ef8c-4bd1-9660-cf4cf37641f0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-8215205d-ef8c-4bd1-9660-cf4cf37641f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:31.804114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:31.805325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:31.806488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:31.806488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:31.806488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:31.806488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:31.806488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:31.806488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:31.806488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:31.806488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:31.806488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:31.806488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:31.812140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 339/689] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 09:45:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:31.813880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf61a2-04df-4522-ba1d-ed482245b214 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:31.814198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf61a2-04df-4522-ba1d-ed482245b214 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:31.814456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf61a2-04df-4522-ba1d-ed482245b214 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:31.816541 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 340/690] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:31.817632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf61a2-04df-4522-ba1d-ed482245b214 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:31.866978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fb1ed-8180-4612-aac6-e67069d814a3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:31.912827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7566ff0-3d80-4c0e-86ad-39e94367024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:31.913662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7566ff0-3d80-4c0e-86ad-39e94367024c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:31.913942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7566ff0-3d80-4c0e-86ad-39e94367024c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:31.914220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7566ff0-3d80-4c0e-86ad-39e94367024c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:31.926644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7566ff0-3d80-4c0e-86ad-39e94367024c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:31.928156 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7566ff0-3d80-4c0e-86ad-39e94367024c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-e7566ff0-3d80-4c0e-86ad-39e94367024c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:31.944199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:31.945204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:31.946142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:31.946142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:31.946142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:31.946142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:31.946142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:31.946142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:31.946142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:31.946142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:31.946142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:31.946142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:31.946821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 351/691] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:31.951140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3f777-b258-4205-a336-3a9fae04ae45 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:31.951140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3f777-b258-4205-a336-3a9fae04ae45 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:31.951140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3f777-b258-4205-a336-3a9fae04ae45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:31.958477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf61a2-04df-4522-ba1d-ed482245b214 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:31.971196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-bdd3f777-b258-4205-a336-3a9fae04ae45 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:31.972446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bdd3f777-b258-4205-a336-3a9fae04ae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:32.009328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fbfb7-4806-4d9c-b48b-43112d488268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:32.010020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fbfb7-4806-4d9c-b48b-43112d488268 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:32.010300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fbfb7-4806-4d9c-b48b-43112d488268 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:32.010644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fbfb7-4806-4d9c-b48b-43112d488268 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:32.011452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fbfb7-4806-4d9c-b48b-43112d488268 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:32.011763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fbfb7-4806-4d9c-b48b-43112d488268 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:32.013228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fbfb7-4806-4d9c-b48b-43112d488268 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:32.035534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:32.042838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:32.047595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 341/692] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 09:45:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:32.048771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fbfb7-4806-4d9c-b48b-43112d488268 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:32.075890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 352/693] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:32.082633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3f777-b258-4205-a336-3a9fae04ae45 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:32.207790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fbfb7-4806-4d9c-b48b-43112d488268 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:32.221526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3f777-b258-4205-a336-3a9fae04ae45 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:32.255182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850cbc7-15fa-42c5-a6e5-96a3944dd7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:32.256551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850cbc7-15fa-42c5-a6e5-96a3944dd7dc None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:32.257276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850cbc7-15fa-42c5-a6e5-96a3944dd7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:32.257698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850cbc7-15fa-42c5-a6e5-96a3944dd7dc None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:32.258727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850cbc7-15fa-42c5-a6e5-96a3944dd7dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:32.259195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850cbc7-15fa-42c5-a6e5-96a3944dd7dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:32.260774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850cbc7-15fa-42c5-a6e5-96a3944dd7dc None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:32.261365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c6d11-d918-4d0b-bc81-49b1996cd744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:32.262134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c6d11-d918-4d0b-bc81-49b1996cd744 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:32.262514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c6d11-d918-4d0b-bc81-49b1996cd744 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:32.262948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c6d11-d918-4d0b-bc81-49b1996cd744 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:32.284086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 342/694] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 09:45:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:32.285281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850cbc7-15fa-42c5-a6e5-96a3944dd7dc None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:32.322846 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:32.356486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c6d11-d918-4d0b-bc81-49b1996cd744 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:32.358817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c6d11-d918-4d0b-bc81-49b1996cd744 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:32.372256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:32.428045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850cbc7-15fa-42c5-a6e5-96a3944dd7dc None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:32.486616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 353/695] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:32.486616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c6d11-d918-4d0b-bc81-49b1996cd744 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:32.491372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878e1ae-7fcf-4e28-bba2-f5ce7d642191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:32.492170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878e1ae-7fcf-4e28-bba2-f5ce7d642191 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:32.492463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878e1ae-7fcf-4e28-bba2-f5ce7d642191 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:32.492726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878e1ae-7fcf-4e28-bba2-f5ce7d642191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:32.511718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878e1ae-7fcf-4e28-bba2-f5ce7d642191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:32.514162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878e1ae-7fcf-4e28-bba2-f5ce7d642191 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-9878e1ae-7fcf-4e28-bba2-f5ce7d642191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:32.537539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:32.538500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:32.539411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:32.539411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:32.539411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:32.539411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:32.543003 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 343/696] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:32.544328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878e1ae-7fcf-4e28-bba2-f5ce7d642191 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:32.581525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:32.610197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c6d11-d918-4d0b-bc81-49b1996cd744 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:32.647801 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15741ad-2574-4aa8-9c0b-e57378b6bb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:32.648480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15741ad-2574-4aa8-9c0b-e57378b6bb2f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:32.648761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15741ad-2574-4aa8-9c0b-e57378b6bb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:32.649033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15741ad-2574-4aa8-9c0b-e57378b6bb2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:32.666341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878e1ae-7fcf-4e28-bba2-f5ce7d642191 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:32.681165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:32.708594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0a5cd-51ef-4170-a858-0486a8d3b3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:32.709975 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0a5cd-51ef-4170-a858-0486a8d3b3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:32.710480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0a5cd-51ef-4170-a858-0486a8d3b3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:32.711002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0a5cd-51ef-4170-a858-0486a8d3b3c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:32.723170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15741ad-2574-4aa8-9c0b-e57378b6bb2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:32.725005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15741ad-2574-4aa8-9c0b-e57378b6bb2f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:32.725724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0a5cd-51ef-4170-a858-0486a8d3b3c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:32.727698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0a5cd-51ef-4170-a858-0486a8d3b3c6 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-04e0a5cd-51ef-4170-a858-0486a8d3b3c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:32.748645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:32.749607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:32.750507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:32.750507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:32.750507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:32.750507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:32.758078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 344/697] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:32.759387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0a5cd-51ef-4170-a858-0486a8d3b3c6 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:32.802807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:32.831069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 354/698] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:32.832006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15741ad-2574-4aa8-9c0b-e57378b6bb2f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:32.878906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0a5cd-51ef-4170-a858-0486a8d3b3c6 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:32.921926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edde71f-56d1-4072-b177-86291db20e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:32.923452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde71f-56d1-4072-b177-86291db20e9e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:32.924736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde71f-56d1-4072-b177-86291db20e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:32.925231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde71f-56d1-4072-b177-86291db20e9e None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:32.926272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edde71f-56d1-4072-b177-86291db20e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:32.926888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edde71f-56d1-4072-b177-86291db20e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:32.928514 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edde71f-56d1-4072-b177-86291db20e9e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:32.946715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 345/699] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:32.947592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edde71f-56d1-4072-b177-86291db20e9e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:32.961738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15741ad-2574-4aa8-9c0b-e57378b6bb2f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:32.984106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:32.999941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ca122-a3e5-4719-8bba-a1331650fa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:33.000687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ca122-a3e5-4719-8bba-a1331650fa70 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:33.001405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ca122-a3e5-4719-8bba-a1331650fa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:33.001405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ca122-a3e5-4719-8bba-a1331650fa70 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:33.002136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ca122-a3e5-4719-8bba-a1331650fa70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:33.002955 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ca122-a3e5-4719-8bba-a1331650fa70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:33.003939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ca122-a3e5-4719-8bba-a1331650fa70 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:33.022065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 355/700] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:33.022065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ca122-a3e5-4719-8bba-a1331650fa70 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:33.073670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:33.104675 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edde71f-56d1-4072-b177-86291db20e9e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:33.155390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28878a-bf01-4ca5-8baa-1acfe8c018e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:33.156218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28878a-bf01-4ca5-8baa-1acfe8c018e6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:33.156500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28878a-bf01-4ca5-8baa-1acfe8c018e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:33.156783 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28878a-bf01-4ca5-8baa-1acfe8c018e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:33.170167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ca122-a3e5-4719-8bba-a1331650fa70 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:33.173878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28878a-bf01-4ca5-8baa-1acfe8c018e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:33.175763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28878a-bf01-4ca5-8baa-1acfe8c018e6 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-2f28878a-bf01-4ca5-8baa-1acfe8c018e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:33.194521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:33.195713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:33.197172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:33.197172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:33.197172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:33.197172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:33.197172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 346/701] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:33.197172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28878a-bf01-4ca5-8baa-1acfe8c018e6 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:33.224197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77cd2a-ef2b-453f-9ef9-c81c9b18ed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:33.225561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77cd2a-ef2b-453f-9ef9-c81c9b18ed32 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:33.226036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77cd2a-ef2b-453f-9ef9-c81c9b18ed32 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:33.226762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77cd2a-ef2b-453f-9ef9-c81c9b18ed32 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:33.228236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77cd2a-ef2b-453f-9ef9-c81c9b18ed32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:33.230256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77cd2a-ef2b-453f-9ef9-c81c9b18ed32 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:33.261931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 356/702] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:45:33 2022] POST /identity/v3/users => generated 254 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:45:33.263209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77cd2a-ef2b-453f-9ef9-c81c9b18ed32 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:33.322936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:33.390042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28878a-bf01-4ca5-8baa-1acfe8c018e6 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:33.471323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:33.477186 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ae248-2ee3-4265-8ae4-aaadd3e36725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:33.478100 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ae248-2ee3-4265-8ae4-aaadd3e36725 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:33.478320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ae248-2ee3-4265-8ae4-aaadd3e36725 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:33.478658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ae248-2ee3-4265-8ae4-aaadd3e36725 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:33.487619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ae248-2ee3-4265-8ae4-aaadd3e36725 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:33.488093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77cd2a-ef2b-453f-9ef9-c81c9b18ed32 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:33.489358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ae248-2ee3-4265-8ae4-aaadd3e36725 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-6d3ae248-2ee3-4265-8ae4-aaadd3e36725 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:33.507093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:33.508383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:33.509491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:33.509491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:33.509491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:33.509491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:33.509491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:33.509491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:33.509491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:33.509491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:33.509491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 347/703] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:33.509960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9d15c-9731-4564-9d81-7cc35439fa59 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:33.510287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9d15c-9731-4564-9d81-7cc35439fa59 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:33.510440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9d15c-9731-4564-9d81-7cc35439fa59 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:33.513116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 348/704] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:33.518605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9d15c-9731-4564-9d81-7cc35439fa59 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:33.538369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565676a-3a5d-4459-b572-4db7a3fcb1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:33.539519 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565676a-3a5d-4459-b572-4db7a3fcb1be None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:33.540001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565676a-3a5d-4459-b572-4db7a3fcb1be None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:33.540494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565676a-3a5d-4459-b572-4db7a3fcb1be None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:33.541346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565676a-3a5d-4459-b572-4db7a3fcb1be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:33.541718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565676a-3a5d-4459-b572-4db7a3fcb1be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:33.543197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565676a-3a5d-4459-b572-4db7a3fcb1be None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:33.557494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 357/705] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:33.558954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd168b-c9eb-47e4-bd49-14671738461f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:33.559241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd168b-c9eb-47e4-bd49-14671738461f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:33.559514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd168b-c9eb-47e4-bd49-14671738461f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:33.579840 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e5cd168b-c9eb-47e4-bd49-14671738461f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:33.581054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e5cd168b-c9eb-47e4-bd49-14671738461f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:33.643522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9d15c-9731-4564-9d81-7cc35439fa59 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:33.670699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 358/706] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:33.672050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd168b-c9eb-47e4-bd49-14671738461f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:33.700465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edccb07-9406-4120-a0f8-0f76e68a58b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:33.701504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edccb07-9406-4120-a0f8-0f76e68a58b3 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:33.701788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edccb07-9406-4120-a0f8-0f76e68a58b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:33.702188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edccb07-9406-4120-a0f8-0f76e68a58b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:33.703436 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edccb07-9406-4120-a0f8-0f76e68a58b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:33.703751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edccb07-9406-4120-a0f8-0f76e68a58b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:33.705905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edccb07-9406-4120-a0f8-0f76e68a58b3 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:33.722916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:33.738590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 349/707] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:33.738590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edccb07-9406-4120-a0f8-0f76e68a58b3 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:33.799008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:33.818997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd168b-c9eb-47e4-bd49-14671738461f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:33.868511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5257c-44ad-425c-986f-5ce1a8bbaf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:33.869658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5257c-44ad-425c-986f-5ce1a8bbaf46 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:33.869942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5257c-44ad-425c-986f-5ce1a8bbaf46 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:33.870219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5257c-44ad-425c-986f-5ce1a8bbaf46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:33.880369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5257c-44ad-425c-986f-5ce1a8bbaf46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:33.882685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5257c-44ad-425c-986f-5ce1a8bbaf46 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:33.885027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edccb07-9406-4120-a0f8-0f76e68a58b3 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-c0f5257c-44ad-425c-986f-5ce1a8bbaf46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:33.901645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:33.902591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:33.903478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:33.903478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:33.903478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:33.903478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:33.903478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:33.903478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:33.903478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:33.903478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:33.903902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 359/708] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:33.904692 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5257c-44ad-425c-986f-5ce1a8bbaf46 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:33.934552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394ee32-2e46-4220-95bd-e49fa31d5b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:33.935890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394ee32-2e46-4220-95bd-e49fa31d5b53 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:33.936265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394ee32-2e46-4220-95bd-e49fa31d5b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:33.936648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394ee32-2e46-4220-95bd-e49fa31d5b53 None admin] PATH_INFO: `/v3/services/network` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:33.937475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394ee32-2e46-4220-95bd-e49fa31d5b53 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:33.939735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394ee32-2e46-4220-95bd-e49fa31d5b53 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-3394ee32-2e46-4220-95bd-e49fa31d5b53 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:33.958441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:33.960763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 09:45:33.961925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:33.966701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 350/709] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:33 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:33.966701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394ee32-2e46-4220-95bd-e49fa31d5b53 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:34.026665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5257c-44ad-425c-986f-5ce1a8bbaf46 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:34.050496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:34.074983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35bab8-5fe5-4066-949d-bf3fcaf8a513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:34.075660 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35bab8-5fe5-4066-949d-bf3fcaf8a513 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:34.075939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35bab8-5fe5-4066-949d-bf3fcaf8a513 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:34.076211 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35bab8-5fe5-4066-949d-bf3fcaf8a513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:34.077065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35bab8-5fe5-4066-949d-bf3fcaf8a513 None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=8a5eea7f539048c48ed10e72dd933df9) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:34.077373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35bab8-5fe5-4066-949d-bf3fcaf8a513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:34.078872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35bab8-5fe5-4066-949d-bf3fcaf8a513 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:34.094597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 360/710] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:33 2022] GET /identity/v3/role_assignments?user.id=8a5eea7f539048c48ed10e72dd933df9&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:34.094597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35bab8-5fe5-4066-949d-bf3fcaf8a513 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:34.103331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394ee32-2e46-4220-95bd-e49fa31d5b53 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:34.122976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:34.163832 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e04c60-5502-460e-a6e0-6a23ddf3905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:34.165221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e04c60-5502-460e-a6e0-6a23ddf3905c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:34.165389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e04c60-5502-460e-a6e0-6a23ddf3905c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:34.165645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e04c60-5502-460e-a6e0-6a23ddf3905c None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:34.166629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e04c60-5502-460e-a6e0-6a23ddf3905c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:34.166948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e04c60-5502-460e-a6e0-6a23ddf3905c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:34.168559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e04c60-5502-460e-a6e0-6a23ddf3905c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:34.186371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 351/711] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:34.188028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102dca-0cfd-452e-8dd2-41fc540d0749 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:34.188377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102dca-0cfd-452e-8dd2-41fc540d0749 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:34.188804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102dca-0cfd-452e-8dd2-41fc540d0749 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:34.190672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 352/712] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:34.193388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7102dca-0cfd-452e-8dd2-41fc540d0749 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:34.242115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35bab8-5fe5-4066-949d-bf3fcaf8a513 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:34.281880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1ada1-e543-40c4-a434-0710560b4377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:34.283122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1ada1-e543-40c4-a434-0710560b4377 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:34.283576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1ada1-e543-40c4-a434-0710560b4377 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:34.283996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1ada1-e543-40c4-a434-0710560b4377 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:34.285166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1ada1-e543-40c4-a434-0710560b4377 None admin] RBAC: Adding query filter params (name=network) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:34.285621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1ada1-e543-40c4-a434-0710560b4377 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:34.287202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1ada1-e543-40c4-a434-0710560b4377 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:34.293260 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 361/713] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:45:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:34.294868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c4f76-e9e6-4884-91d8-23d0dda72bb9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:34.295279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c4f76-e9e6-4884-91d8-23d0dda72bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:34.295675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c4f76-e9e6-4884-91d8-23d0dda72bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:34.313015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9e5c4f76-e9e6-4884-91d8-23d0dda72bb9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:34.314782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9e5c4f76-e9e6-4884-91d8-23d0dda72bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:34.354749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7102dca-0cfd-452e-8dd2-41fc540d0749 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:34.385918 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:34.398117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:34.410690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 362/714] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:34.410690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c4f76-e9e6-4884-91d8-23d0dda72bb9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:34.416369 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6da1fb-27ab-401c-bbe9-247948f71559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:34.417327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6da1fb-27ab-401c-bbe9-247948f71559 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:34.417783 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6da1fb-27ab-401c-bbe9-247948f71559 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:34.418222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6da1fb-27ab-401c-bbe9-247948f71559 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:34.419350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6da1fb-27ab-401c-bbe9-247948f71559 None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=d86d073ef5ee4631b508ab578662be96) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:34.419875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6da1fb-27ab-401c-bbe9-247948f71559 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:34.421613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6da1fb-27ab-401c-bbe9-247948f71559 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:34.436109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 353/715] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:34 2022] GET /identity/v3/role_assignments?user.id=d86d073ef5ee4631b508ab578662be96&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:34.444602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6da1fb-27ab-401c-bbe9-247948f71559 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:34.546548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c4f76-e9e6-4884-91d8-23d0dda72bb9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:34.597254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab84d54-26e4-42c3-8209-b29ebd35b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:34.597953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab84d54-26e4-42c3-8209-b29ebd35b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:34.598236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab84d54-26e4-42c3-8209-b29ebd35b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:34.598513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab84d54-26e4-42c3-8209-b29ebd35b5a6 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:34.600635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab84d54-26e4-42c3-8209-b29ebd35b5a6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:34.600956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab84d54-26e4-42c3-8209-b29ebd35b5a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:34.602399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab84d54-26e4-42c3-8209-b29ebd35b5a6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:34.609999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 363/716] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:45:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:34.678127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6da1fb-27ab-401c-bbe9-247948f71559 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:34.730592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d5481-50af-4c2c-97f2-50ecaac33501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:34.731473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d5481-50af-4c2c-97f2-50ecaac33501 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:34.731754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d5481-50af-4c2c-97f2-50ecaac33501 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:34.732027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d5481-50af-4c2c-97f2-50ecaac33501 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:34.732707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d5481-50af-4c2c-97f2-50ecaac33501 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:34.734119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d5481-50af-4c2c-97f2-50ecaac33501 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-586d5481-50af-4c2c-97f2-50ecaac33501 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:34.748743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:34.749746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:34.750628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 354/717] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Jun 29 09:45:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:34.758548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab84d54-26e4-42c3-8209-b29ebd35b5a6 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:34.841795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:34.895506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab84d54-26e4-42c3-8209-b29ebd35b5a6 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:34.935747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f133d3-97aa-4913-8b87-003ed4b8d69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:34.936444 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f133d3-97aa-4913-8b87-003ed4b8d69e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:34.936728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f133d3-97aa-4913-8b87-003ed4b8d69e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:34.937007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f133d3-97aa-4913-8b87-003ed4b8d69e None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:34.937905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f133d3-97aa-4913-8b87-003ed4b8d69e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:34.938216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f133d3-97aa-4913-8b87-003ed4b8d69e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:34.939936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f133d3-97aa-4913-8b87-003ed4b8d69e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:34.945319 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 364/718] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 09:45:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:34.950769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d5481-50af-4c2c-97f2-50ecaac33501 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:35.027891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:35.110424 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d5481-50af-4c2c-97f2-50ecaac33501 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:35.152801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25441a0-a6d1-4e6f-9493-85db89a4254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:35.153483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25441a0-a6d1-4e6f-9493-85db89a4254f None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:35.153765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25441a0-a6d1-4e6f-9493-85db89a4254f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:35.154040 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25441a0-a6d1-4e6f-9493-85db89a4254f None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:35.154791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25441a0-a6d1-4e6f-9493-85db89a4254f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:35.155101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25441a0-a6d1-4e6f-9493-85db89a4254f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:35.156507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25441a0-a6d1-4e6f-9493-85db89a4254f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:35.163417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 355/719] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 09:45:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:35.869265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98860771-3ddd-4e3d-88f3-6a63680ddd1a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:35.869994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98860771-3ddd-4e3d-88f3-6a63680ddd1a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:35.869994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98860771-3ddd-4e3d-88f3-6a63680ddd1a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:35.871559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 365/720] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:35.884448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d4161-336f-4b47-854f-bf04e2c73e60 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:35.885072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d4161-336f-4b47-854f-bf04e2c73e60 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:35.885072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d4161-336f-4b47-854f-bf04e2c73e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:35.906783 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-662d4161-336f-4b47-854f-bf04e2c73e60 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:35.908556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-662d4161-336f-4b47-854f-bf04e2c73e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:35.986283 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 356/721] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:36.030256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98860771-3ddd-4e3d-88f3-6a63680ddd1a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:36.171740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98860771-3ddd-4e3d-88f3-6a63680ddd1a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:36.206121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:36.222320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05201015-3ad6-4b48-88b7-895d50648061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:36.223174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05201015-3ad6-4b48-88b7-895d50648061 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:36.223455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05201015-3ad6-4b48-88b7-895d50648061 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:36.223726 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05201015-3ad6-4b48-88b7-895d50648061 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:36.247504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05201015-3ad6-4b48-88b7-895d50648061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-05201015-3ad6-4b48-88b7-895d50648061 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:36.255399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:36.257520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:36.259500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:36.266598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 366/722] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:36.266862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d4161-336f-4b47-854f-bf04e2c73e60 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:36.330988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:36.415417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d4161-336f-4b47-854f-bf04e2c73e60 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:36.428813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41d3cd-c73f-40da-879f-510796a66abd None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:36.428988 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41d3cd-c73f-40da-879f-510796a66abd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:36.429270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41d3cd-c73f-40da-879f-510796a66abd None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:36.431029 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 367/723] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:36.436418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6183d1c-2b26-4c14-91a0-9ceb2637ca68 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:36.436706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6183d1c-2b26-4c14-91a0-9ceb2637ca68 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:36.437038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6183d1c-2b26-4c14-91a0-9ceb2637ca68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:36.463146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c6183d1c-2b26-4c14-91a0-9ceb2637ca68 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:36.464375 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c6183d1c-2b26-4c14-91a0-9ceb2637ca68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:36.481098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f93864-fe75-42d0-8b46-ae1418aa0eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:36.481796 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f93864-fe75-42d0-8b46-ae1418aa0eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:36.482073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f93864-fe75-42d0-8b46-ae1418aa0eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:36.482347 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f93864-fe75-42d0-8b46-ae1418aa0eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:36.483179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f93864-fe75-42d0-8b46-ae1418aa0eb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:36.483483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f93864-fe75-42d0-8b46-ae1418aa0eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:36.484880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f93864-fe75-42d0-8b46-ae1418aa0eb0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:36.502004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 357/724] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:36.506130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f93864-fe75-42d0-8b46-ae1418aa0eb0 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:36.606675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 368/725] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:36.641804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6183d1c-2b26-4c14-91a0-9ceb2637ca68 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:36.704258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f93864-fe75-42d0-8b46-ae1418aa0eb0 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:36.706328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:36.748501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8b18e-a37b-48f0-b369-3da1b465cb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:36.750185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8b18e-a37b-48f0-b369-3da1b465cb7c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:36.750978 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8b18e-a37b-48f0-b369-3da1b465cb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:36.751543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8b18e-a37b-48f0-b369-3da1b465cb7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:36.762237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6183d1c-2b26-4c14-91a0-9ceb2637ca68 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:36.767573 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8b18e-a37b-48f0-b369-3da1b465cb7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:36.769307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8b18e-a37b-48f0-b369-3da1b465cb7c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:36.800109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-02f8b18e-a37b-48f0-b369-3da1b465cb7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:36.808457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:36.809414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:36.811027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:36.811027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:36.811027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:36.811027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:36.811027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 358/726] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:36.817162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8b18e-a37b-48f0-b369-3da1b465cb7c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:36.840020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3103098-01aa-4115-aac7-d199184efbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:36.841245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3103098-01aa-4115-aac7-d199184efbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:36.841722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3103098-01aa-4115-aac7-d199184efbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:36.842225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3103098-01aa-4115-aac7-d199184efbc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:36.864903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3103098-01aa-4115-aac7-d199184efbc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-a3103098-01aa-4115-aac7-d199184efbc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:36.875505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:36.877050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:36.877922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:36.882165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 369/727] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:36.888071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3103098-01aa-4115-aac7-d199184efbc2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:37.035989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3103098-01aa-4115-aac7-d199184efbc2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:37.080530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc546e7-2f3c-45d2-9792-ad638b7c559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:37.081702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc546e7-2f3c-45d2-9792-ad638b7c559f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.081812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8b18e-a37b-48f0-b369-3da1b465cb7c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:37.082743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc546e7-2f3c-45d2-9792-ad638b7c559f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.083241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc546e7-2f3c-45d2-9792-ad638b7c559f None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.084244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc546e7-2f3c-45d2-9792-ad638b7c559f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:37.084754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc546e7-2f3c-45d2-9792-ad638b7c559f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:37.086380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc546e7-2f3c-45d2-9792-ad638b7c559f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:37.096812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 370/728] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:37.102321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc546e7-2f3c-45d2-9792-ad638b7c559f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:37.151282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:37.175373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc58be8-cad4-4ba8-a3e0-db764dae7107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:37.176084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc58be8-cad4-4ba8-a3e0-db764dae7107 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.176368 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc58be8-cad4-4ba8-a3e0-db764dae7107 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.176644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc58be8-cad4-4ba8-a3e0-db764dae7107 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.177404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc58be8-cad4-4ba8-a3e0-db764dae7107 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:37.177716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc58be8-cad4-4ba8-a3e0-db764dae7107 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:37.179432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc58be8-cad4-4ba8-a3e0-db764dae7107 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:37.207412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 359/729] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:37.209325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd23f92-058c-4fb9-97d4-90ba2333d9a7 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.209761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd23f92-058c-4fb9-97d4-90ba2333d9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.210054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd23f92-058c-4fb9-97d4-90ba2333d9a7 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.212020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 360/730] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:37.213608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a7248-2de3-41d9-8b3a-7dd9a2782c11 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.213899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a7248-2de3-41d9-8b3a-7dd9a2782c11 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.214181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a7248-2de3-41d9-8b3a-7dd9a2782c11 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.215792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 361/731] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:37.216547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a7248-2de3-41d9-8b3a-7dd9a2782c11 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:37.234690 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:37.234894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc546e7-2f3c-45d2-9792-ad638b7c559f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:37.270923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4d02d-ec93-4916-b26a-5e220fc0dca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:37.271748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4d02d-ec93-4916-b26a-5e220fc0dca5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.272028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4d02d-ec93-4916-b26a-5e220fc0dca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.272302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4d02d-ec93-4916-b26a-5e220fc0dca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.284241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4d02d-ec93-4916-b26a-5e220fc0dca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:37.285774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4d02d-ec93-4916-b26a-5e220fc0dca5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-a7f4d02d-ec93-4916-b26a-5e220fc0dca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:37.305510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:37.306618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:37.307534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:37.307534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:37.307534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:37.307534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:37.307534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 371/732] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:37.308394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b41a8-c916-4170-8e66-418cdc7d1d17 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.308675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b41a8-c916-4170-8e66-418cdc7d1d17 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.308948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b41a8-c916-4170-8e66-418cdc7d1d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.327853 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-085b41a8-c916-4170-8e66-418cdc7d1d17 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:37.329109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-085b41a8-c916-4170-8e66-418cdc7d1d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:37.340663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a7248-2de3-41d9-8b3a-7dd9a2782c11 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:37.384483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d527f-f667-49ba-acaf-4ba7e837a50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:37.385312 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d527f-f667-49ba-acaf-4ba7e837a50a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.385593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d527f-f667-49ba-acaf-4ba7e837a50a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.385866 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d527f-f667-49ba-acaf-4ba7e837a50a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.394313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d527f-f667-49ba-acaf-4ba7e837a50a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:37.396079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d527f-f667-49ba-acaf-4ba7e837a50a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:37.407001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 372/733] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:37.408658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35f9ff-ef9d-4292-8aaa-d738005c9740 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.408941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35f9ff-ef9d-4292-8aaa-d738005c9740 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.409211 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35f9ff-ef9d-4292-8aaa-d738005c9740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-938d527f-f667-49ba-acaf-4ba7e837a50a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:37.423693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:37.424739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:37.425664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:37.425664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:37.425664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:37.425664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:37.425664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:37.425664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:37.425664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:37.425664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:37.425664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:45:37.425664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:37.432804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 362/734] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 09:45:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:37.433754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d527f-f667-49ba-acaf-4ba7e837a50a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:37.436903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-bb35f9ff-ef9d-4292-8aaa-d738005c9740 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:37.438082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bb35f9ff-ef9d-4292-8aaa-d738005c9740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:37.455337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:37.517837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:37.518178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 373/735] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:37.519087 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35f9ff-ef9d-4292-8aaa-d738005c9740 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:37.559521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d527f-f667-49ba-acaf-4ba7e837a50a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:37.602617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477df7fa-f093-4558-90ec-5a98d5f3f6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:37.603301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477df7fa-f093-4558-90ec-5a98d5f3f6fd None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.603586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477df7fa-f093-4558-90ec-5a98d5f3f6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.603863 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477df7fa-f093-4558-90ec-5a98d5f3f6fd None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.604596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477df7fa-f093-4558-90ec-5a98d5f3f6fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:37.604905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477df7fa-f093-4558-90ec-5a98d5f3f6fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:37.606301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477df7fa-f093-4558-90ec-5a98d5f3f6fd None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:37.619405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 363/736] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:37.620271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477df7fa-f093-4558-90ec-5a98d5f3f6fd None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:37.643302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35f9ff-ef9d-4292-8aaa-d738005c9740 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:37.683150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d46032-6c63-478a-a6ab-bf0a426fbc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:37.683831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d46032-6c63-478a-a6ab-bf0a426fbc7a None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.684111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d46032-6c63-478a-a6ab-bf0a426fbc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.684385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d46032-6c63-478a-a6ab-bf0a426fbc7a None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.685243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d46032-6c63-478a-a6ab-bf0a426fbc7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:37.686886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d46032-6c63-478a-a6ab-bf0a426fbc7a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:37.716249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 374/737] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:45:37 2022] POST /identity/v3/services => generated 240 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:45:37.716249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d46032-6c63-478a-a6ab-bf0a426fbc7a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:37.735154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:37.739529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477df7fa-f093-4558-90ec-5a98d5f3f6fd None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:37.803145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:37.815497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304702fe-7b98-4706-8bd8-ff26d5d49e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:37.816255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304702fe-7b98-4706-8bd8-ff26d5d49e19 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.817185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304702fe-7b98-4706-8bd8-ff26d5d49e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.817520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304702fe-7b98-4706-8bd8-ff26d5d49e19 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.818462 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304702fe-7b98-4706-8bd8-ff26d5d49e19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:37.818865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304702fe-7b98-4706-8bd8-ff26d5d49e19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:37.821302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304702fe-7b98-4706-8bd8-ff26d5d49e19 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:37.848188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d46032-6c63-478a-a6ab-bf0a426fbc7a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:37.854275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 364/738] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 09:45:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:37.856519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304702fe-7b98-4706-8bd8-ff26d5d49e19 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:37.892597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72a183-aecb-4094-9458-4e76df037bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:37.894153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a183-aecb-4094-9458-4e76df037bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:37.894699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a183-aecb-4094-9458-4e76df037bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:37.895224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a183-aecb-4094-9458-4e76df037bc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:37.908366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72a183-aecb-4094-9458-4e76df037bc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-9d72a183-aecb-4094-9458-4e76df037bc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:37.917014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:37.917993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 375/739] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:37.919322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72a183-aecb-4094-9458-4e76df037bc8 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:37.992404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304702fe-7b98-4706-8bd8-ff26d5d49e19 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:38.012235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:38.033674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ea383-3ecd-4d71-9c64-8acccd2b3fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:38.034488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ea383-3ecd-4d71-9c64-8acccd2b3fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:38.034803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ea383-3ecd-4d71-9c64-8acccd2b3fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:38.035623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ea383-3ecd-4d71-9c64-8acccd2b3fb5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:38.045410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ea383-3ecd-4d71-9c64-8acccd2b3fb5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:38.047600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ea383-3ecd-4d71-9c64-8acccd2b3fb5 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:38.053007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72a183-aecb-4094-9458-4e76df037bc8 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-cf2ea383-3ecd-4d71-9c64-8acccd2b3fb5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:38.064559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:38.065763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 365/740] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 09:45:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:38.066724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ea383-3ecd-4d71-9c64-8acccd2b3fb5 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:38.094528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75025504-ab02-42a7-8d2d-bf22ae25b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:38.095333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75025504-ab02-42a7-8d2d-bf22ae25b42a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:38.095643 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75025504-ab02-42a7-8d2d-bf22ae25b42a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:38.095871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75025504-ab02-42a7-8d2d-bf22ae25b42a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:38.125831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:38.176168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75025504-ab02-42a7-8d2d-bf22ae25b42a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:38.178046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75025504-ab02-42a7-8d2d-bf22ae25b42a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:38.183257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ea383-3ecd-4d71-9c64-8acccd2b3fb5 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:38.227240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47210e4-ad2e-4cde-9395-4966730d8bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:38.228609 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47210e4-ad2e-4cde-9395-4966730d8bbd None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:38.229102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47210e4-ad2e-4cde-9395-4966730d8bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:38.229732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47210e4-ad2e-4cde-9395-4966730d8bbd None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:38.231071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47210e4-ad2e-4cde-9395-4966730d8bbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:38.231621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47210e4-ad2e-4cde-9395-4966730d8bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:38.233820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47210e4-ad2e-4cde-9395-4966730d8bbd None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:38.245087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 366/741] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:38.247242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c37b1-4d1c-428b-8f8c-e0a4d9e379db None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:38.247754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c37b1-4d1c-428b-8f8c-e0a4d9e379db None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:38.248800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c37b1-4d1c-428b-8f8c-e0a4d9e379db None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:38.250832 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 367/742] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:38.253218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c37b1-4d1c-428b-8f8c-e0a4d9e379db None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:38.258027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:38.291746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 376/743] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:37 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 09:45:38.292726 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75025504-ab02-42a7-8d2d-bf22ae25b42a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:38.377830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c37b1-4d1c-428b-8f8c-e0a4d9e379db None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:38.391040 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:38.406738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75025504-ab02-42a7-8d2d-bf22ae25b42a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:38.420427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfa4b6-446f-416f-9fce-b9672587d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:38.421124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfa4b6-446f-416f-9fce-b9672587d897 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:38.421404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfa4b6-446f-416f-9fce-b9672587d897 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:38.421680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfa4b6-446f-416f-9fce-b9672587d897 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:38.422532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfa4b6-446f-416f-9fce-b9672587d897 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:38.422927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfa4b6-446f-416f-9fce-b9672587d897 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:38.424483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfa4b6-446f-416f-9fce-b9672587d897 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:38.445036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 368/744] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 09:45:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:38.446387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e5953-2553-4329-b773-02053d1ea1b2 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:38.446720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e5953-2553-4329-b773-02053d1ea1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:38.447022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e5953-2553-4329-b773-02053d1ea1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:38.453216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5876361-9b98-49fb-bfff-318e84957548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:38.454513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5876361-9b98-49fb-bfff-318e84957548 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:38.454911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5876361-9b98-49fb-bfff-318e84957548 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:38.455173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5876361-9b98-49fb-bfff-318e84957548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:38.464098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-e89e5953-2553-4329-b773-02053d1ea1b2 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:38.465291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e89e5953-2553-4329-b773-02053d1ea1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:38.465749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5876361-9b98-49fb-bfff-318e84957548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:38.467280 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5876361-9b98-49fb-bfff-318e84957548 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-a5876361-9b98-49fb-bfff-318e84957548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:38.486108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:38.487147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:38.488028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:38.488028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:38.488028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:38.488028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:38.492119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 377/745] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:38.492741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5876361-9b98-49fb-bfff-318e84957548 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:38.520196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:38.550586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 369/746] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:38 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 09:45:38.550586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e5953-2553-4329-b773-02053d1ea1b2 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:38.613408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5876361-9b98-49fb-bfff-318e84957548 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:38.649776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6bb68-4b10-4db1-969e-60d7ea69ac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:38.650504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6bb68-4b10-4db1-969e-60d7ea69ac3f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:38.650816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6bb68-4b10-4db1-969e-60d7ea69ac3f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:38.651089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6bb68-4b10-4db1-969e-60d7ea69ac3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:38.656181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:38.660982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e5953-2553-4329-b773-02053d1ea1b2 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:38.662721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6bb68-4b10-4db1-969e-60d7ea69ac3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:38.664223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6bb68-4b10-4db1-969e-60d7ea69ac3f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f1a6bb68-4b10-4db1-969e-60d7ea69ac3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:38.681902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:38.682871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:38.683741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:38.683741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:38.683741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:38.683741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:38.687169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 378/747] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:38.688825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6bb68-4b10-4db1-969e-60d7ea69ac3f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:38.700218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f803113-fadf-4d61-9f8d-525d9653512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:38.701023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f803113-fadf-4d61-9f8d-525d9653512b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:38.701268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f803113-fadf-4d61-9f8d-525d9653512b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:38.701568 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f803113-fadf-4d61-9f8d-525d9653512b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:38.712500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f803113-fadf-4d61-9f8d-525d9653512b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:38.714009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f803113-fadf-4d61-9f8d-525d9653512b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-4f803113-fadf-4d61-9f8d-525d9653512b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:38.732867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:38.733795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:38.734722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:38.734722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:38.734722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:38.734722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:38.739366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 370/748] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:38.739971 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f803113-fadf-4d61-9f8d-525d9653512b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:38.775132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:38.801099 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6bb68-4b10-4db1-969e-60d7ea69ac3f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:38.836270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b099037-6926-46ef-aed1-66552b8dc1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:38.836940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b099037-6926-46ef-aed1-66552b8dc1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:38.837219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b099037-6926-46ef-aed1-66552b8dc1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:38.837494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b099037-6926-46ef-aed1-66552b8dc1c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:38.838222 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b099037-6926-46ef-aed1-66552b8dc1c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:38.838587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b099037-6926-46ef-aed1-66552b8dc1c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:38.839987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b099037-6926-46ef-aed1-66552b8dc1c8 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:38.853871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 379/749] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:38.854639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b099037-6926-46ef-aed1-66552b8dc1c8 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:38.859729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f803113-fadf-4d61-9f8d-525d9653512b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:38.900026 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c1c29-264c-4428-8e2f-d6bc13b2b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:38.900595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c1c29-264c-4428-8e2f-d6bc13b2b34b None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:38.900871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c1c29-264c-4428-8e2f-d6bc13b2b34b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:38.901140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c1c29-264c-4428-8e2f-d6bc13b2b34b None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:38.901862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c1c29-264c-4428-8e2f-d6bc13b2b34b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:38.903262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c1c29-264c-4428-8e2f-d6bc13b2b34b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:38.930610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 371/750] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:45:38 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:45:38.930610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c1c29-264c-4428-8e2f-d6bc13b2b34b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:38.944674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:38.997452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b099037-6926-46ef-aed1-66552b8dc1c8 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:39.056531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c1c29-264c-4428-8e2f-d6bc13b2b34b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:39.063096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebd72e-1fce-4803-b7f6-267ba6545b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:39.065577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebd72e-1fce-4803-b7f6-267ba6545b72 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:39.066082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebd72e-1fce-4803-b7f6-267ba6545b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:39.066600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebd72e-1fce-4803-b7f6-267ba6545b72 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:39.068030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebd72e-1fce-4803-b7f6-267ba6545b72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:39.068986 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebd72e-1fce-4803-b7f6-267ba6545b72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:39.070954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebd72e-1fce-4803-b7f6-267ba6545b72 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:39.073929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:39.095852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 380/751] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:39.097958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebd72e-1fce-4803-b7f6-267ba6545b72 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:39.112015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9424d9-da1b-49a4-a698-6f2e0c33d820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:39.112771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9424d9-da1b-49a4-a698-6f2e0c33d820 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:39.113052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9424d9-da1b-49a4-a698-6f2e0c33d820 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:39.113374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9424d9-da1b-49a4-a698-6f2e0c33d820 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:39.114173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9424d9-da1b-49a4-a698-6f2e0c33d820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:39.114487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9424d9-da1b-49a4-a698-6f2e0c33d820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:39.115933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9424d9-da1b-49a4-a698-6f2e0c33d820 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:39.131155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 372/752] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:39.132810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9424d9-da1b-49a4-a698-6f2e0c33d820 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:39.227008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:39.230525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebd72e-1fce-4803-b7f6-267ba6545b72 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:39.266926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9424d9-da1b-49a4-a698-6f2e0c33d820 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:39.267279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcb378-c951-4793-a12b-97369870df30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:39.268101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcb378-c951-4793-a12b-97369870df30 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:39.268372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcb378-c951-4793-a12b-97369870df30 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:39.268640 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcb378-c951-4793-a12b-97369870df30 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:39.277685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcb378-c951-4793-a12b-97369870df30 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:39.279206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcb378-c951-4793-a12b-97369870df30 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-20bcb378-c951-4793-a12b-97369870df30 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:39.294245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:39.295291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:39.296233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:39.296233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:39.296233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:39.296233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:39.296233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:39.296233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:39.296233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:39.296233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:39.296233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:39.296233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:39.296233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 381/753] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:39.297847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcb378-c951-4793-a12b-97369870df30 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:39.319797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d00af-e505-4fb7-a5ef-10927c10ec30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:39.321030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d00af-e505-4fb7-a5ef-10927c10ec30 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:39.321521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d00af-e505-4fb7-a5ef-10927c10ec30 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:39.321970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d00af-e505-4fb7-a5ef-10927c10ec30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:39.331861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d00af-e505-4fb7-a5ef-10927c10ec30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:39.334816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d00af-e505-4fb7-a5ef-10927c10ec30 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-901d00af-e505-4fb7-a5ef-10927c10ec30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:39.351354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:39.352309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:39.353195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:39.353195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:39.353195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:39.353195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:39.353195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:39.353195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:39.353195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:39.353195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:39.353195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 373/754] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:39.353862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d00af-e505-4fb7-a5ef-10927c10ec30 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:39.365143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:39.426129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcb378-c951-4793-a12b-97369870df30 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:39.463440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d00af-e505-4fb7-a5ef-10927c10ec30 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:39.476909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd50d08-611e-4aab-a287-2e26fbb4fe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:39.478368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd50d08-611e-4aab-a287-2e26fbb4fe61 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:39.478841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd50d08-611e-4aab-a287-2e26fbb4fe61 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:39.479429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd50d08-611e-4aab-a287-2e26fbb4fe61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:39.490154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd50d08-611e-4aab-a287-2e26fbb4fe61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:39.492604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd50d08-611e-4aab-a287-2e26fbb4fe61 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:39.495021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:39.508632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53d019-d747-4ebd-b07c-3094d05db632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:39.509472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53d019-d747-4ebd-b07c-3094d05db632 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:39.509849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53d019-d747-4ebd-b07c-3094d05db632 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:39.510219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53d019-d747-4ebd-b07c-3094d05db632 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:39.511220 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53d019-d747-4ebd-b07c-3094d05db632 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:39.511629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53d019-d747-4ebd-b07c-3094d05db632 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-5cd50d08-611e-4aab-a287-2e26fbb4fe61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:39.512617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:39.513666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:39.514656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:39.514656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:39.514656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:39.514656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:39.514656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:39.514656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:39.514656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:39.514656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:39.515099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53d019-d747-4ebd-b07c-3094d05db632 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:39.517504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 382/755] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:39.518646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd50d08-611e-4aab-a287-2e26fbb4fe61 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:39.541489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 374/756] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 09:45:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:39.550218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53d019-d747-4ebd-b07c-3094d05db632 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:39.634917 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd50d08-611e-4aab-a287-2e26fbb4fe61 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:39.651748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:39.665659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53d019-d747-4ebd-b07c-3094d05db632 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:39.677699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ab0df-e74e-43a5-b509-4b866c55a53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:39.678376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ab0df-e74e-43a5-b509-4b866c55a53d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:39.678702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ab0df-e74e-43a5-b509-4b866c55a53d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:39.678980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ab0df-e74e-43a5-b509-4b866c55a53d None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:39.680011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ab0df-e74e-43a5-b509-4b866c55a53d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:39.680210 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ab0df-e74e-43a5-b509-4b866c55a53d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:39.682048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ab0df-e74e-43a5-b509-4b866c55a53d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:39.700664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 383/757] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:39.710622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ab0df-e74e-43a5-b509-4b866c55a53d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:39.714545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccfb8b-b330-4832-ac65-418cf17753c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:39.715607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccfb8b-b330-4832-ac65-418cf17753c0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:39.716113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccfb8b-b330-4832-ac65-418cf17753c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:39.716559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccfb8b-b330-4832-ac65-418cf17753c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:39.717867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccfb8b-b330-4832-ac65-418cf17753c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:39.719446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccfb8b-b330-4832-ac65-418cf17753c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:39.721970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccfb8b-b330-4832-ac65-418cf17753c0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:39.739076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 375/758] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:39.740266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccfb8b-b330-4832-ac65-418cf17753c0 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:39.793680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:39.838875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ab0df-e74e-43a5-b509-4b866c55a53d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:39.861943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccfb8b-b330-4832-ac65-418cf17753c0 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:39.878730 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a7c5b-9826-4480-be01-1bccb6194725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:39.879377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a7c5b-9826-4480-be01-1bccb6194725 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:39.879618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a7c5b-9826-4480-be01-1bccb6194725 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:39.879915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a7c5b-9826-4480-be01-1bccb6194725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:39.900477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669a2ee-f2e5-4116-a573-603886b05370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:39.901148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669a2ee-f2e5-4116-a573-603886b05370 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:39.901430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669a2ee-f2e5-4116-a573-603886b05370 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:39.902131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669a2ee-f2e5-4116-a573-603886b05370 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:39.903019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669a2ee-f2e5-4116-a573-603886b05370 None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=a9ac6a36e5224f38bdc5b214580ff18a) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:39.903335 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669a2ee-f2e5-4116-a573-603886b05370 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:39.904783 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669a2ee-f2e5-4116-a573-603886b05370 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:39.911151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 376/759] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:39 2022] GET /identity/v3/role_assignments?user.id=a9ac6a36e5224f38bdc5b214580ff18a&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:39.912221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669a2ee-f2e5-4116-a573-603886b05370 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:39.940781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:39.978526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a7c5b-9826-4480-be01-1bccb6194725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:39.980642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a7c5b-9826-4480-be01-1bccb6194725 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:40.058446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669a2ee-f2e5-4116-a573-603886b05370 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:40.071453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:40.104820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e931b9-14d5-4c56-b5fc-ee7c351076be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:40.106607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e931b9-14d5-4c56-b5fc-ee7c351076be None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:40.107460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e931b9-14d5-4c56-b5fc-ee7c351076be None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:40.107937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e931b9-14d5-4c56-b5fc-ee7c351076be None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/8a5eea7f539048c48ed10e72dd933df9/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:40.109839 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 384/760] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:45:40.111395 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c23429-33fb-47eb-be0a-6db6c671237c None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:40.111686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c23429-33fb-47eb-be0a-6db6c671237c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:40.111964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c23429-33fb-47eb-be0a-6db6c671237c None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:40.112332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e931b9-14d5-4c56-b5fc-ee7c351076be None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=8a5eea7f539048c48ed10e72dd933df9, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:40.113738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 385/761] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:40.115377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e931b9-14d5-4c56-b5fc-ee7c351076be None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:40.115525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c23429-33fb-47eb-be0a-6db6c671237c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:40.139909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 377/762] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:45:39 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/8a5eea7f539048c48ed10e72dd933df9/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:45:40.142257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec253-bf02-4d93-8785-cf36ede37076 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:40.143071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec253-bf02-4d93-8785-cf36ede37076 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:40.143731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec253-bf02-4d93-8785-cf36ede37076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:40.168391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-641ec253-bf02-4d93-8785-cf36ede37076 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:40.170233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-641ec253-bf02-4d93-8785-cf36ede37076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:40.258459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 378/763] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:40.281566 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ec253-bf02-4d93-8785-cf36ede37076 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:40.295378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:40.343144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c23429-33fb-47eb-be0a-6db6c671237c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:40.398154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3ffe7-c66d-4109-8398-6c0142ee0da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:40.399286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3ffe7-c66d-4109-8398-6c0142ee0da5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:40.399568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3ffe7-c66d-4109-8398-6c0142ee0da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:40.399845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3ffe7-c66d-4109-8398-6c0142ee0da5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:40.413207 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3ffe7-c66d-4109-8398-6c0142ee0da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:40.414741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3ffe7-c66d-4109-8398-6c0142ee0da5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-a2e3ffe7-c66d-4109-8398-6c0142ee0da5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:40.433822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:40.434835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:40.435733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:40.435733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:40.435733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:40.435733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:40.435951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:40.436023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 386/764] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:40.438503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ec253-bf02-4d93-8785-cf36ede37076 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:40.443265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3ffe7-c66d-4109-8398-6c0142ee0da5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:40.505028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cd6e6-d0f0-45a5-b53a-500c8343b9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:40.506805 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cd6e6-d0f0-45a5-b53a-500c8343b9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:40.507312 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cd6e6-d0f0-45a5-b53a-500c8343b9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:40.507793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cd6e6-d0f0-45a5-b53a-500c8343b9c2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:40.509005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0cd6e6-d0f0-45a5-b53a-500c8343b9c2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:40.512174 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0cd6e6-d0f0-45a5-b53a-500c8343b9c2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-3a0cd6e6-d0f0-45a5-b53a-500c8343b9c2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:40.528803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:40.531168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:40.533191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 379/765] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:40 2022] GET /identity/v3/services/network => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:40.542170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cd6e6-d0f0-45a5-b53a-500c8343b9c2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:40.683897 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3ffe7-c66d-4109-8398-6c0142ee0da5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:40.744755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:40.761543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343421d-7ece-4b0f-a55c-a53600b50f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:40.762249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343421d-7ece-4b0f-a55c-a53600b50f30 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:40.762533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343421d-7ece-4b0f-a55c-a53600b50f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:40.762839 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343421d-7ece-4b0f-a55c-a53600b50f30 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:40.763595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343421d-7ece-4b0f-a55c-a53600b50f30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:40.763905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343421d-7ece-4b0f-a55c-a53600b50f30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:40.765384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343421d-7ece-4b0f-a55c-a53600b50f30 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:40.789287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 387/766] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:40.795904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343421d-7ece-4b0f-a55c-a53600b50f30 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:40.806540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cd6e6-d0f0-45a5-b53a-500c8343b9c2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:40.860912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb44333-f7cc-4dc6-a916-a766fcd7d345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:40.861614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb44333-f7cc-4dc6-a916-a766fcd7d345 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:40.861896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb44333-f7cc-4dc6-a916-a766fcd7d345 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:40.862171 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb44333-f7cc-4dc6-a916-a766fcd7d345 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:40.862927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb44333-f7cc-4dc6-a916-a766fcd7d345 None admin] RBAC: Adding query filter params (name=network) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:40.863239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb44333-f7cc-4dc6-a916-a766fcd7d345 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:40.864647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb44333-f7cc-4dc6-a916-a766fcd7d345 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:40.880275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 380/767] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:45:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:40.883629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb44333-f7cc-4dc6-a916-a766fcd7d345 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:40.917559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:41.005073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343421d-7ece-4b0f-a55c-a53600b50f30 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:41.066476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb44333-f7cc-4dc6-a916-a766fcd7d345 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:41.091337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3dd18-47c9-453a-a7df-69e55f4b6547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:41.092273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3dd18-47c9-453a-a7df-69e55f4b6547 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:41.092555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3dd18-47c9-453a-a7df-69e55f4b6547 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:41.092827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3dd18-47c9-453a-a7df-69e55f4b6547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:41.106049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3dd18-47c9-453a-a7df-69e55f4b6547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:41.107570 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3dd18-47c9-453a-a7df-69e55f4b6547 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:41.111238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfbc27-6124-4c88-a7d9-d01570bd97e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:41.111932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfbc27-6124-4c88-a7d9-d01570bd97e0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:41.112218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfbc27-6124-4c88-a7d9-d01570bd97e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:41.112497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfbc27-6124-4c88-a7d9-d01570bd97e0 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:41.113229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfbc27-6124-4c88-a7d9-d01570bd97e0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:41.113542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfbc27-6124-4c88-a7d9-d01570bd97e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:41.114956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfbc27-6124-4c88-a7d9-d01570bd97e0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-3bf3dd18-47c9-453a-a7df-69e55f4b6547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:41.122803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:41.124021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:41.124901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:41.124901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:41.124901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:41.124901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:41.124901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:41.124901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:41.124901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:41.124901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:41.128931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 388/768] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:41.130166 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 381/769] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:45:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:41.130351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3dd18-47c9-453a-a7df-69e55f4b6547 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:41.135674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfbc27-6124-4c88-a7d9-d01570bd97e0 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:41.183288 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:41.243242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3dd18-47c9-453a-a7df-69e55f4b6547 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:41.280931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4494e-45bf-4c8a-947d-b98b1876e448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:41.281526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4494e-45bf-4c8a-947d-b98b1876e448 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:41.281812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4494e-45bf-4c8a-947d-b98b1876e448 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:41.282166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4494e-45bf-4c8a-947d-b98b1876e448 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:41.283013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4494e-45bf-4c8a-947d-b98b1876e448 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:41.283326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4494e-45bf-4c8a-947d-b98b1876e448 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:41.284814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4494e-45bf-4c8a-947d-b98b1876e448 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:41.304103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 389/770] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:41.305408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4494e-45bf-4c8a-947d-b98b1876e448 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:41.341281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:41.362684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfbc27-6124-4c88-a7d9-d01570bd97e0 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:41.408003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4494e-45bf-4c8a-947d-b98b1876e448 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:41.433268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a988ea-7340-41fa-858c-0741b57c8a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:41.434011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a988ea-7340-41fa-858c-0741b57c8a6b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:41.434296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a988ea-7340-41fa-858c-0741b57c8a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:41.434584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a988ea-7340-41fa-858c-0741b57c8a6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:41.435391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a988ea-7340-41fa-858c-0741b57c8a6b None admin] RBAC: Adding query filter params (interface=public, service_id=806865cc45604412b183fa8798acb600, region_id=RegionOne) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:41.435698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a988ea-7340-41fa-858c-0741b57c8a6b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:41.437072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a988ea-7340-41fa-858c-0741b57c8a6b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:41.446930 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:41.451350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb235ac9-5f69-41e0-9268-302214546358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:41.451824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 382/771] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 09:45:41 2022] GET /identity/v3/endpoints?service_id=806865cc45604412b183fa8798acb600&interface=public®ion_id=RegionOne => generated 194 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:41.452818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb235ac9-5f69-41e0-9268-302214546358 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:41.453130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb235ac9-5f69-41e0-9268-302214546358 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:41.456439 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb235ac9-5f69-41e0-9268-302214546358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:41.457328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb235ac9-5f69-41e0-9268-302214546358 None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=d86d073ef5ee4631b508ab578662be96) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:41.457528 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a988ea-7340-41fa-858c-0741b57c8a6b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:41.457645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb235ac9-5f69-41e0-9268-302214546358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:41.459181 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb235ac9-5f69-41e0-9268-302214546358 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:41.471009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 390/772] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:41 2022] GET /identity/v3/role_assignments?user.id=d86d073ef5ee4631b508ab578662be96&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:41.473131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300733d-5ecd-4ebc-98d8-df27b220af59 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:41.473430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300733d-5ecd-4ebc-98d8-df27b220af59 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:41.474075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300733d-5ecd-4ebc-98d8-df27b220af59 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:41.479473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 391/773] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:41.481528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9e5de-33aa-4245-9f01-078885819645 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:41.481821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9e5de-33aa-4245-9f01-078885819645 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:41.482100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9e5de-33aa-4245-9f01-078885819645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:41.503186 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a5f9e5de-33aa-4245-9f01-078885819645 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:41.504757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a5f9e5de-33aa-4245-9f01-078885819645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:41.580795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a988ea-7340-41fa-858c-0741b57c8a6b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:41.584722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 392/774] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:45:41.644520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c646b51-377c-4a34-874d-b989dbab27ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:41.645619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c646b51-377c-4a34-874d-b989dbab27ee None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:41.646097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c646b51-377c-4a34-874d-b989dbab27ee None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:41.646614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c646b51-377c-4a34-874d-b989dbab27ee None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:41.647529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c646b51-377c-4a34-874d-b989dbab27ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:41.649265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c646b51-377c-4a34-874d-b989dbab27ee None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:41.663134 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 383/775] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 09:45:41 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:41.671824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9e5de-33aa-4245-9f01-078885819645 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:41.804734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:41.807034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9e5de-33aa-4245-9f01-078885819645 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:41.853587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf5429-3b3e-4561-bef3-96100c6f862f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:41.854873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf5429-3b3e-4561-bef3-96100c6f862f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:41.855328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf5429-3b3e-4561-bef3-96100c6f862f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:41.855793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf5429-3b3e-4561-bef3-96100c6f862f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:41.856691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf5429-3b3e-4561-bef3-96100c6f862f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:41.859999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf5429-3b3e-4561-bef3-96100c6f862f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-23cf5429-3b3e-4561-bef3-96100c6f862f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:41.878739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:41.879779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 09:45:41.880614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:41.883921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 393/776] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Jun 29 09:45:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:41.890452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c646b51-377c-4a34-874d-b989dbab27ee None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:41.915810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:42.000457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c646b51-377c-4a34-874d-b989dbab27ee None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:42.038421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d9b7d-4f8c-4b08-a86d-284ecaba5c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:42.039162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d9b7d-4f8c-4b08-a86d-284ecaba5c3a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:42.039441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d9b7d-4f8c-4b08-a86d-284ecaba5c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:42.039710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d9b7d-4f8c-4b08-a86d-284ecaba5c3a None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:42.040444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d9b7d-4f8c-4b08-a86d-284ecaba5c3a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:42.040751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d9b7d-4f8c-4b08-a86d-284ecaba5c3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:42.042150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d9b7d-4f8c-4b08-a86d-284ecaba5c3a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:42.048022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 384/777] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 09:45:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:42.055665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf5429-3b3e-4561-bef3-96100c6f862f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:42.180682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf5429-3b3e-4561-bef3-96100c6f862f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:42.215321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0684dc2-bf52-4918-8b5d-36bf71e9a918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:42.216005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0684dc2-bf52-4918-8b5d-36bf71e9a918 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:42.216282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0684dc2-bf52-4918-8b5d-36bf71e9a918 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:42.216552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0684dc2-bf52-4918-8b5d-36bf71e9a918 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:42.217268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0684dc2-bf52-4918-8b5d-36bf71e9a918 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:42.217570 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0684dc2-bf52-4918-8b5d-36bf71e9a918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:42.218987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0684dc2-bf52-4918-8b5d-36bf71e9a918 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:42.225742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 394/778] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 09:45:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:42.231531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d9b7d-4f8c-4b08-a86d-284ecaba5c3a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:42.289531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a085c5-edb0-4607-9934-1a2f1bae39a0 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:42.290082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a085c5-edb0-4607-9934-1a2f1bae39a0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:42.290361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a085c5-edb0-4607-9934-1a2f1bae39a0 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:42.292517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 395/779] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:45:42.298231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863e9e5-c900-455c-baeb-cd97c77ddf73 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:42.298519 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863e9e5-c900-455c-baeb-cd97c77ddf73 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:42.298827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863e9e5-c900-455c-baeb-cd97c77ddf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:42.323152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:42.335750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d863e9e5-c900-455c-baeb-cd97c77ddf73 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:42.336956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d863e9e5-c900-455c-baeb-cd97c77ddf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:42.348347 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d9b7d-4f8c-4b08-a86d-284ecaba5c3a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:42.391916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e020f2-e273-47f6-8c0d-a13d60c389ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:42.393025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e020f2-e273-47f6-8c0d-a13d60c389ae None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:42.393508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e020f2-e273-47f6-8c0d-a13d60c389ae None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:42.393983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e020f2-e273-47f6-8c0d-a13d60c389ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:42.396557 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e020f2-e273-47f6-8c0d-a13d60c389ae None admin] RBAC: Adding query filter params (interface=public, service_id=05cae293db844762986699923b6d75e1, region_id=RegionOne) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:42.397143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e020f2-e273-47f6-8c0d-a13d60c389ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:42.398781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e020f2-e273-47f6-8c0d-a13d60c389ae None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:42.398891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:42.409850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 385/780] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 09:45:42 2022] GET /identity/v3/endpoints?service_id=05cae293db844762986699923b6d75e1&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:42.419955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e020f2-e273-47f6-8c0d-a13d60c389ae None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:42.495636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 396/781] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:45:42.544882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863e9e5-c900-455c-baeb-cd97c77ddf73 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:42.590994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e020f2-e273-47f6-8c0d-a13d60c389ae None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:42.635270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2ae3c-83d7-4026-b53d-c6e64e145578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:42.636232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2ae3c-83d7-4026-b53d-c6e64e145578 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:42.636639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2ae3c-83d7-4026-b53d-c6e64e145578 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:42.637038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2ae3c-83d7-4026-b53d-c6e64e145578 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:42.638081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2ae3c-83d7-4026-b53d-c6e64e145578 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:42.643071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2ae3c-83d7-4026-b53d-c6e64e145578 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:42.650735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 386/782] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 09:45:42 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:42.652125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8364b-0aa0-4d29-849a-e7d17997f703 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:42.652412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8364b-0aa0-4d29-849a-e7d17997f703 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:42.652688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8364b-0aa0-4d29-849a-e7d17997f703 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:42.654302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 387/783] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:42.660881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880366dd-597e-4d20-ad64-d9197db6ac93 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:42.661383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880366dd-597e-4d20-ad64-d9197db6ac93 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:42.661875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880366dd-597e-4d20-ad64-d9197db6ac93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:42.705388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-880366dd-597e-4d20-ad64-d9197db6ac93 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:42.707640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-880366dd-597e-4d20-ad64-d9197db6ac93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:42.724790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:42.824685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863e9e5-c900-455c-baeb-cd97c77ddf73 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:42.900093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 388/784] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:42.913009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c5103-bed8-4cfe-a383-2651b482ebc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:42.913877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c5103-bed8-4cfe-a383-2651b482ebc8 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:42.915078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c5103-bed8-4cfe-a383-2651b482ebc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:42.915372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c5103-bed8-4cfe-a383-2651b482ebc8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:42.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c5103-bed8-4cfe-a383-2651b482ebc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:42.931822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880366dd-597e-4d20-ad64-d9197db6ac93 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-376c5103-bed8-4cfe-a383-2651b482ebc8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:42.937808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:42.939021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:42.940080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 397/785] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:42.947347 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c5103-bed8-4cfe-a383-2651b482ebc8 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:42.968970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:43.068713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880366dd-597e-4d20-ad64-d9197db6ac93 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:43.097130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c5103-bed8-4cfe-a383-2651b482ebc8 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:43.114643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1a878-cf64-4f0e-82f4-dead5d32592c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:43.114643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1a878-cf64-4f0e-82f4-dead5d32592c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:43.114643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1a878-cf64-4f0e-82f4-dead5d32592c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:43.114643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1a878-cf64-4f0e-82f4-dead5d32592c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:43.123217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:43.127490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1a878-cf64-4f0e-82f4-dead5d32592c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-83d1a878-cf64-4f0e-82f4-dead5d32592c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:43.137302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:43.139265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:43.141042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:43.142380 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 389/786] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:43.152833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1a878-cf64-4f0e-82f4-dead5d32592c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:43.229455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb211-1a2e-4166-a63a-bbccbec9a941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:43.231582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb211-1a2e-4166-a63a-bbccbec9a941 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:43.232336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb211-1a2e-4166-a63a-bbccbec9a941 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:43.233129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb211-1a2e-4166-a63a-bbccbec9a941 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:43.234741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74feb211-1a2e-4166-a63a-bbccbec9a941 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:43.235604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74feb211-1a2e-4166-a63a-bbccbec9a941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:43.246662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74feb211-1a2e-4166-a63a-bbccbec9a941 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:43.268682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 398/787] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:43.272866 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1a878-cf64-4f0e-82f4-dead5d32592c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:43.274732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb211-1a2e-4166-a63a-bbccbec9a941 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:43.312536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5d8fd-ee8d-42af-9816-5e3d124809d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:43.313257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5d8fd-ee8d-42af-9816-5e3d124809d2 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:43.313564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5d8fd-ee8d-42af-9816-5e3d124809d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:43.313815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5d8fd-ee8d-42af-9816-5e3d124809d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:43.314600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5d8fd-ee8d-42af-9816-5e3d124809d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:43.314969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5d8fd-ee8d-42af-9816-5e3d124809d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:43.316430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5d8fd-ee8d-42af-9816-5e3d124809d2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:43.327346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 390/788] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:43.343071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5d8fd-ee8d-42af-9816-5e3d124809d2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:43.416985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:43.461572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5d8fd-ee8d-42af-9816-5e3d124809d2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:43.516700 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7edf20-f083-4ffa-a1bc-3b7eb6615306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:43.517542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7edf20-f083-4ffa-a1bc-3b7eb6615306 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:43.517831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7edf20-f083-4ffa-a1bc-3b7eb6615306 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:43.518113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7edf20-f083-4ffa-a1bc-3b7eb6615306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:43.548559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7edf20-f083-4ffa-a1bc-3b7eb6615306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:43.550205 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7edf20-f083-4ffa-a1bc-3b7eb6615306 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-de7edf20-f083-4ffa-a1bc-3b7eb6615306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:43.579520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:43.581814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:43.583471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:43.583471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:43.583471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:43.583471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:43.583471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 391/789] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:43.592954 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7edf20-f083-4ffa-a1bc-3b7eb6615306 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:43.721505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:43.722916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7edf20-f083-4ffa-a1bc-3b7eb6615306 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:43.747699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:43.757392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb211-1a2e-4166-a63a-bbccbec9a941 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:43.784672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504250b-cf3f-461c-85bc-7034c89f7d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:43.785355 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504250b-cf3f-461c-85bc-7034c89f7d66 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:43.785630 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504250b-cf3f-461c-85bc-7034c89f7d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:43.785888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504250b-cf3f-461c-85bc-7034c89f7d66 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:43.786643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504250b-cf3f-461c-85bc-7034c89f7d66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:43.787238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504250b-cf3f-461c-85bc-7034c89f7d66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:43.788627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504250b-cf3f-461c-85bc-7034c89f7d66 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:43.804760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b310e-d80b-4dc0-8ded-8f411b9853cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:43.805574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b310e-d80b-4dc0-8ded-8f411b9853cb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:43.805839 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b310e-d80b-4dc0-8ded-8f411b9853cb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:43.806106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b310e-d80b-4dc0-8ded-8f411b9853cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:43.821292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 392/790] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:43.825293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504250b-cf3f-461c-85bc-7034c89f7d66 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:43.831084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b310e-d80b-4dc0-8ded-8f411b9853cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:43.832562 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b310e-d80b-4dc0-8ded-8f411b9853cb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f18b310e-d80b-4dc0-8ded-8f411b9853cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:43.863440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:43.865067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:43.866679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:43.866679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:43.866679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:43.866679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:43.866679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 399/791] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 591 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:43.872505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b310e-d80b-4dc0-8ded-8f411b9853cb None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:44.007962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504250b-cf3f-461c-85bc-7034c89f7d66 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:44.047511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b310e-d80b-4dc0-8ded-8f411b9853cb None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:44.048595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b7eb9-2910-499f-aae8-e010999b0e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:44.049449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b7eb9-2910-499f-aae8-e010999b0e42 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:44.049911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b7eb9-2910-499f-aae8-e010999b0e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:44.050349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b7eb9-2910-499f-aae8-e010999b0e42 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:44.060895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b7eb9-2910-499f-aae8-e010999b0e42 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:44.062365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b7eb9-2910-499f-aae8-e010999b0e42 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-0a6b7eb9-2910-499f-aae8-e010999b0e42 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:44.077727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:44.078792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:44.079801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:44.079801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:44.079801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:44.079801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:44.079801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:44.079801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:44.079801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:44.079801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:44.079801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:45:44.079801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:44.079801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 393/792] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 09:45:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:44.081051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a996a6-4004-475a-8216-c5667a0e2a63 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:44.081051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a996a6-4004-475a-8216-c5667a0e2a63 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:44.081202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a996a6-4004-475a-8216-c5667a0e2a63 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:44.086881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 394/793] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:44.087739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a996a6-4004-475a-8216-c5667a0e2a63 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:44.091278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:44.100150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb0abc-2c52-4949-ac3b-b6438bd36fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:44.101740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb0abc-2c52-4949-ac3b-b6438bd36fce None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:44.102677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb0abc-2c52-4949-ac3b-b6438bd36fce None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:44.103151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb0abc-2c52-4949-ac3b-b6438bd36fce None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:44.104461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb0abc-2c52-4949-ac3b-b6438bd36fce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:44.104993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb0abc-2c52-4949-ac3b-b6438bd36fce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:44.107624 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb0abc-2c52-4949-ac3b-b6438bd36fce None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:44.126506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 400/794] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:44.127441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:44.130699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f229d-859f-46da-b1e1-7ede1d1fa4e1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:44.131265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f229d-859f-46da-b1e1-7ede1d1fa4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:44.131700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f229d-859f-46da-b1e1-7ede1d1fa4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:44.151310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-589f229d-859f-46da-b1e1-7ede1d1fa4e1 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:44.152922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-589f229d-859f-46da-b1e1-7ede1d1fa4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:44.211323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a996a6-4004-475a-8216-c5667a0e2a63 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:44.242394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 401/795] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:44.243488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f229d-859f-46da-b1e1-7ede1d1fa4e1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:44.254435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdbb9e-5a24-4f74-b722-5d80fea0c4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:44.255474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdbb9e-5a24-4f74-b722-5d80fea0c4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:44.255967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdbb9e-5a24-4f74-b722-5d80fea0c4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:44.256450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdbb9e-5a24-4f74-b722-5d80fea0c4d1 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:44.257482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfdbb9e-5a24-4f74-b722-5d80fea0c4d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:44.258038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfdbb9e-5a24-4f74-b722-5d80fea0c4d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:44.259710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfdbb9e-5a24-4f74-b722-5d80fea0c4d1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:44.279236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 395/796] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 09:45:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:44.280441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdbb9e-5a24-4f74-b722-5d80fea0c4d1 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:44.381790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:44.383446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f229d-859f-46da-b1e1-7ede1d1fa4e1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:44.392413 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdbb9e-5a24-4f74-b722-5d80fea0c4d1 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:44.408316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:44.421614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6297bd-551d-4b27-9049-970a3ef684a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:44.422710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6297bd-551d-4b27-9049-970a3ef684a4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:44.423172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6297bd-551d-4b27-9049-970a3ef684a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:44.423530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6297bd-551d-4b27-9049-970a3ef684a4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:44.431939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08d2c5-1ef0-4937-a680-6a42dff10e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:44.432496 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6297bd-551d-4b27-9049-970a3ef684a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:44.433468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6297bd-551d-4b27-9049-970a3ef684a4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:44.433755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08d2c5-1ef0-4937-a680-6a42dff10e58 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:44.434214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08d2c5-1ef0-4937-a680-6a42dff10e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:44.434694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08d2c5-1ef0-4937-a680-6a42dff10e58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:44.445745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08d2c5-1ef0-4937-a680-6a42dff10e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-ef6297bd-551d-4b27-9049-970a3ef684a4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:44.449302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:44.450199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:44.451169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:44.451169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:44.451169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:44.451169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:44.451169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:44.451169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:44.451169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:44.451169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:44.451169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:45:44.451169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:44.451169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 402/797] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 09:45:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:44.451744 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6297bd-551d-4b27-9049-970a3ef684a4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-bb08d2c5-1ef0-4937-a680-6a42dff10e58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:44.456787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:44.457699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:44.458543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:44.459199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 396/798] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:44.461463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ed89a-9c7e-484a-be90-38dbf9f5d228 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:44.461768 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ed89a-9c7e-484a-be90-38dbf9f5d228 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:44.462041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ed89a-9c7e-484a-be90-38dbf9f5d228 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:44.466923 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 397/799] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:44.468058 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ed89a-9c7e-484a-be90-38dbf9f5d228 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:44.579965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6297bd-551d-4b27-9049-970a3ef684a4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:44.619065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864508c-6ca3-4703-8459-07a4acf42eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:44.619735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864508c-6ca3-4703-8459-07a4acf42eff None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:44.620019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864508c-6ca3-4703-8459-07a4acf42eff None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:44.620285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864508c-6ca3-4703-8459-07a4acf42eff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:44.679778 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:44.703965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864508c-6ca3-4703-8459-07a4acf42eff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:44.705660 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864508c-6ca3-4703-8459-07a4acf42eff None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:44.710478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ed89a-9c7e-484a-be90-38dbf9f5d228 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:44.753387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e45ce-90d3-4809-8ac7-3899b4af4299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:44.754092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e45ce-90d3-4809-8ac7-3899b4af4299 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:44.754367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e45ce-90d3-4809-8ac7-3899b4af4299 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:44.754671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e45ce-90d3-4809-8ac7-3899b4af4299 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:44.755530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e45ce-90d3-4809-8ac7-3899b4af4299 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:44.755836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e45ce-90d3-4809-8ac7-3899b4af4299 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:44.757303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e45ce-90d3-4809-8ac7-3899b4af4299 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:44.784096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 398/800] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 09:45:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:44.784096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a6e4a-c60b-48d4-a185-b462ff9e90ea None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:44.784096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a6e4a-c60b-48d4-a185-b462ff9e90ea None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:44.784096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a6e4a-c60b-48d4-a185-b462ff9e90ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:44.800093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-946a6e4a-c60b-48d4-a185-b462ff9e90ea None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:44.801313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-946a6e4a-c60b-48d4-a185-b462ff9e90ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:44.804672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:44.875933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 403/801] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:45:44.875933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864508c-6ca3-4703-8459-07a4acf42eff None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:44.907656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 399/802] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:44.915820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a6e4a-c60b-48d4-a185-b462ff9e90ea None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:44.994696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:45.009638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864508c-6ca3-4703-8459-07a4acf42eff None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.038787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a6e4a-c60b-48d4-a185-b462ff9e90ea None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.055505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660ee68-6cbc-4328-a255-1cf9cec6fc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.056201 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660ee68-6cbc-4328-a255-1cf9cec6fc83 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.056481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660ee68-6cbc-4328-a255-1cf9cec6fc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.056757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660ee68-6cbc-4328-a255-1cf9cec6fc83 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.057520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660ee68-6cbc-4328-a255-1cf9cec6fc83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:45.057827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660ee68-6cbc-4328-a255-1cf9cec6fc83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:45.059281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660ee68-6cbc-4328-a255-1cf9cec6fc83 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:45.073452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 404/803] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:45.073452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660ee68-6cbc-4328-a255-1cf9cec6fc83 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:45.089295 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3b8e9-6b5d-4c54-99b1-15cbb63929be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.090173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3b8e9-6b5d-4c54-99b1-15cbb63929be None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.090454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3b8e9-6b5d-4c54-99b1-15cbb63929be None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.090795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3b8e9-6b5d-4c54-99b1-15cbb63929be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.100203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:45.102212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3b8e9-6b5d-4c54-99b1-15cbb63929be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:45.103924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3b8e9-6b5d-4c54-99b1-15cbb63929be None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-4de3b8e9-6b5d-4c54-99b1-15cbb63929be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:45.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:45.124325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:45.125372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:45.125372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:45.125372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:45.125372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:45.125372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 400/804] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:45.126436 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3b8e9-6b5d-4c54-99b1-15cbb63929be None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:45.190744 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660ee68-6cbc-4328-a255-1cf9cec6fc83 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.229176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00492575-8acb-49f8-91cc-fb46f03ba646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.229979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00492575-8acb-49f8-91cc-fb46f03ba646 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.230253 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00492575-8acb-49f8-91cc-fb46f03ba646 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.230521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00492575-8acb-49f8-91cc-fb46f03ba646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.233457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3b8e9-6b5d-4c54-99b1-15cbb63929be None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.243154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00492575-8acb-49f8-91cc-fb46f03ba646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:45.244643 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00492575-8acb-49f8-91cc-fb46f03ba646 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:45.247540 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-00492575-8acb-49f8-91cc-fb46f03ba646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:45.264399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:45.265521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:45.266523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:45.266523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:45.266523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:45.266523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:45.266523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 405/805] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:45.266948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00492575-8acb-49f8-91cc-fb46f03ba646 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:45.279707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671575c-7ea9-4c8a-8413-0ae65fd5fd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.280763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671575c-7ea9-4c8a-8413-0ae65fd5fd05 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.281048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671575c-7ea9-4c8a-8413-0ae65fd5fd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.281325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671575c-7ea9-4c8a-8413-0ae65fd5fd05 None admin] PATH_INFO: `/v3/services/network` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.282086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671575c-7ea9-4c8a-8413-0ae65fd5fd05 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:45.283494 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671575c-7ea9-4c8a-8413-0ae65fd5fd05 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-3671575c-7ea9-4c8a-8413-0ae65fd5fd05 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:45.296284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:45.297338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 09:45:45.299057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:45.302803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 401/806] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:45 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:45.303723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671575c-7ea9-4c8a-8413-0ae65fd5fd05 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:45.365638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:45.381974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00492575-8acb-49f8-91cc-fb46f03ba646 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.421473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671575c-7ea9-4c8a-8413-0ae65fd5fd05 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.421716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f67b95-b2ee-4b7d-9097-80f7715c9b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.422434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f67b95-b2ee-4b7d-9097-80f7715c9b50 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.422825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f67b95-b2ee-4b7d-9097-80f7715c9b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.423105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f67b95-b2ee-4b7d-9097-80f7715c9b50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.436830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f67b95-b2ee-4b7d-9097-80f7715c9b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:45.438813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f67b95-b2ee-4b7d-9097-80f7715c9b50 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-08f67b95-b2ee-4b7d-9097-80f7715c9b50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:45.457608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:45.459262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:45.461837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:45.461837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:45.461837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:45.461837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:45.461837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 406/807] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:45.461837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f67b95-b2ee-4b7d-9097-80f7715c9b50 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:45.470936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529d53f-40ce-4e52-a8ef-c53c46e61c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.471722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529d53f-40ce-4e52-a8ef-c53c46e61c49 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.471984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529d53f-40ce-4e52-a8ef-c53c46e61c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.472199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529d53f-40ce-4e52-a8ef-c53c46e61c49 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.472906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529d53f-40ce-4e52-a8ef-c53c46e61c49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:45.473224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529d53f-40ce-4e52-a8ef-c53c46e61c49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:45.475009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529d53f-40ce-4e52-a8ef-c53c46e61c49 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:45.497880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 402/808] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:45.497880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529d53f-40ce-4e52-a8ef-c53c46e61c49 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:45.531145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:45.603190 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529d53f-40ce-4e52-a8ef-c53c46e61c49 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.613702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f67b95-b2ee-4b7d-9097-80f7715c9b50 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.625059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:45.648754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ea87c-8671-4c0a-8be7-c87474c6d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.649341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ea87c-8671-4c0a-8be7-c87474c6d4be None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.649605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ea87c-8671-4c0a-8be7-c87474c6d4be None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.649879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ea87c-8671-4c0a-8be7-c87474c6d4be None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.650617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ea87c-8671-4c0a-8be7-c87474c6d4be None admin] RBAC: Adding query filter params (name=network) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:45.651007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ea87c-8671-4c0a-8be7-c87474c6d4be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:45.652323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ea87c-8671-4c0a-8be7-c87474c6d4be None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:45.652542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6801d8-a664-4520-9d88-2a735ddc9028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.653218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6801d8-a664-4520-9d88-2a735ddc9028 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.653497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6801d8-a664-4520-9d88-2a735ddc9028 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.653764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6801d8-a664-4520-9d88-2a735ddc9028 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.654499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6801d8-a664-4520-9d88-2a735ddc9028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:45.654867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6801d8-a664-4520-9d88-2a735ddc9028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:45.656246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6801d8-a664-4520-9d88-2a735ddc9028 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:45.658032 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 403/809] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:45:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:45.658928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ea87c-8671-4c0a-8be7-c87474c6d4be None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:45.676894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 407/810] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:45.676894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6801d8-a664-4520-9d88-2a735ddc9028 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:45.771737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ea87c-8671-4c0a-8be7-c87474c6d4be None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.780537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6801d8-a664-4520-9d88-2a735ddc9028 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.790812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:45.806601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740b2b4-523c-4a0b-9f71-cb22fad90821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.807280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740b2b4-523c-4a0b-9f71-cb22fad90821 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.807559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740b2b4-523c-4a0b-9f71-cb22fad90821 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.807832 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740b2b4-523c-4a0b-9f71-cb22fad90821 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.808560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740b2b4-523c-4a0b-9f71-cb22fad90821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:45.808864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740b2b4-523c-4a0b-9f71-cb22fad90821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:45.810236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740b2b4-523c-4a0b-9f71-cb22fad90821 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:45.816995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e8c9-d4ab-4278-be20-93eaeb0a42b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.817811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e8c9-d4ab-4278-be20-93eaeb0a42b4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.818100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e8c9-d4ab-4278-be20-93eaeb0a42b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.818367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e8c9-d4ab-4278-be20-93eaeb0a42b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.822738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 404/811] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:45.823592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740b2b4-523c-4a0b-9f71-cb22fad90821 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:45.825887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7e8c9-d4ab-4278-be20-93eaeb0a42b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:45.827388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7e8c9-d4ab-4278-be20-93eaeb0a42b4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-bce7e8c9-d4ab-4278-be20-93eaeb0a42b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:45.843527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:45.844794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:45.846189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:45.846189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:45.846189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:45.846189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:45.846189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:45.846189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:45.846189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:45.846189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:45.846189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 408/812] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:45.846189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e8c9-d4ab-4278-be20-93eaeb0a42b4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:45.871364 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:45.930546 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740b2b4-523c-4a0b-9f71-cb22fad90821 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.948052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e8c9-d4ab-4278-be20-93eaeb0a42b4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:45.965933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bca8f5-2576-4ae8-883e-9cfe14e267d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.966640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bca8f5-2576-4ae8-883e-9cfe14e267d9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.966933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bca8f5-2576-4ae8-883e-9cfe14e267d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.967249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bca8f5-2576-4ae8-883e-9cfe14e267d9 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.967972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bca8f5-2576-4ae8-883e-9cfe14e267d9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:45.968264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bca8f5-2576-4ae8-883e-9cfe14e267d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:45.969701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bca8f5-2576-4ae8-883e-9cfe14e267d9 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:45.975756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 405/813] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:45:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:45.977378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4155639-a290-4c6a-9da1-77844b35569d None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.977653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4155639-a290-4c6a-9da1-77844b35569d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.977934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4155639-a290-4c6a-9da1-77844b35569d None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:45.979407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 406/814] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:45.983605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4155639-a290-4c6a-9da1-77844b35569d None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:45.992739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06a634-7b89-4950-ba61-cd7ffe00a691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:45.993649 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06a634-7b89-4950-ba61-cd7ffe00a691 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:45.993967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06a634-7b89-4950-ba61-cd7ffe00a691 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:45.994266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06a634-7b89-4950-ba61-cd7ffe00a691 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:46.002741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06a634-7b89-4950-ba61-cd7ffe00a691 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:46.004230 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06a634-7b89-4950-ba61-cd7ffe00a691 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-9d06a634-7b89-4950-ba61-cd7ffe00a691 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:46.018958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:46.020046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:46.020909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:46.020909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:46.020909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:46.020909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:46.020909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:46.020909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:46.020909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:46.020909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:46.020909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 409/815] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:46.021386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06a634-7b89-4950-ba61-cd7ffe00a691 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:46.052380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:46.108561 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4155639-a290-4c6a-9da1-77844b35569d None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:46.111423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:46.130842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06a634-7b89-4950-ba61-cd7ffe00a691 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:46.145799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057aae0a-7fca-4f8d-94ee-bb120c291e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:46.146631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057aae0a-7fca-4f8d-94ee-bb120c291e13 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:46.146913 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057aae0a-7fca-4f8d-94ee-bb120c291e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:46.147181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057aae0a-7fca-4f8d-94ee-bb120c291e13 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:46.154812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057aae0a-7fca-4f8d-94ee-bb120c291e13 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:46.156293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057aae0a-7fca-4f8d-94ee-bb120c291e13 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:46.167828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e1c38-4f07-4dd9-b4ef-9af995f164eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:46.168506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1c38-4f07-4dd9-b4ef-9af995f164eb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:46.168792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1c38-4f07-4dd9-b4ef-9af995f164eb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:46.169069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1c38-4f07-4dd9-b4ef-9af995f164eb None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:46.169873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e1c38-4f07-4dd9-b4ef-9af995f164eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:46.170186 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e1c38-4f07-4dd9-b4ef-9af995f164eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:46.171686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e1c38-4f07-4dd9-b4ef-9af995f164eb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-057aae0a-7fca-4f8d-94ee-bb120c291e13 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:46.172043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:46.173290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:46.174328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:46.174328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:46.174328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:46.174328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:46.174328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:46.174328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:46.174328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:46.174328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:46.174328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:46.174328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:46.174328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 407/816] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:46.174990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057aae0a-7fca-4f8d-94ee-bb120c291e13 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:46.190183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 410/817] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:46.191676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1d9f4-a163-48dd-8f47-1625bb427cca None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:46.191971 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1d9f4-a163-48dd-8f47-1625bb427cca None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:46.192250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1d9f4-a163-48dd-8f47-1625bb427cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:46.210967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b0a1d9f4-a163-48dd-8f47-1625bb427cca None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:46.212192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b0a1d9f4-a163-48dd-8f47-1625bb427cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:46.282304 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 411/818] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:46.283461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1d9f4-a163-48dd-8f47-1625bb427cca None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:46.291743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057aae0a-7fca-4f8d-94ee-bb120c291e13 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:46.304712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:46.328261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb260e7-8cad-4a8f-b6a7-e2ae7196bf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:46.329017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb260e7-8cad-4a8f-b6a7-e2ae7196bf8a None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:46.329301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb260e7-8cad-4a8f-b6a7-e2ae7196bf8a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:46.329864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb260e7-8cad-4a8f-b6a7-e2ae7196bf8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:46.330299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb260e7-8cad-4a8f-b6a7-e2ae7196bf8a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:46.331714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb260e7-8cad-4a8f-b6a7-e2ae7196bf8a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:46.359160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 408/819] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 09:45:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:45:46.360024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb260e7-8cad-4a8f-b6a7-e2ae7196bf8a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:46.390601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:46.396576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1d9f4-a163-48dd-8f47-1625bb427cca None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:46.449831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb2048-476d-4d1a-a818-0fcf4bf33033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:46.450411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb2048-476d-4d1a-a818-0fcf4bf33033 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:46.450739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb2048-476d-4d1a-a818-0fcf4bf33033 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:46.451020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb2048-476d-4d1a-a818-0fcf4bf33033 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:46.451854 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb2048-476d-4d1a-a818-0fcf4bf33033 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:46.452153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb2048-476d-4d1a-a818-0fcf4bf33033 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:46.453668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb2048-476d-4d1a-a818-0fcf4bf33033 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:46.466687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 412/820] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:46.467533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb2048-476d-4d1a-a818-0fcf4bf33033 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:46.494547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb260e7-8cad-4a8f-b6a7-e2ae7196bf8a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:46.536888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222e3f6-2aae-438f-bb3e-12b2db2d7344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:46.537571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222e3f6-2aae-438f-bb3e-12b2db2d7344 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:46.537851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222e3f6-2aae-438f-bb3e-12b2db2d7344 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:46.538122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222e3f6-2aae-438f-bb3e-12b2db2d7344 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:46.539792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222e3f6-2aae-438f-bb3e-12b2db2d7344 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:46.540131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222e3f6-2aae-438f-bb3e-12b2db2d7344 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:46.541550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222e3f6-2aae-438f-bb3e-12b2db2d7344 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:46.556795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:46.569622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 409/821] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 09:45:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:46.570696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222e3f6-2aae-438f-bb3e-12b2db2d7344 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:46.589291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb2048-476d-4d1a-a818-0fcf4bf33033 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:46.625293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ca1b3-1638-4f48-9282-c14f9f1a0f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:46.626270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ca1b3-1638-4f48-9282-c14f9f1a0f03 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:46.626552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ca1b3-1638-4f48-9282-c14f9f1a0f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:46.626855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ca1b3-1638-4f48-9282-c14f9f1a0f03 None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/a9ac6a36e5224f38bdc5b214580ff18a/roles/9b6eb8ccaac643cc93781d6abe31bb8f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:46.641595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ca1b3-1638-4f48-9282-c14f9f1a0f03 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=a9ac6a36e5224f38bdc5b214580ff18a, role_id=9b6eb8ccaac643cc93781d6abe31bb8f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:46.643728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ca1b3-1638-4f48-9282-c14f9f1a0f03 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:46.661001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 413/822] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:45:46 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/a9ac6a36e5224f38bdc5b214580ff18a/roles/9b6eb8ccaac643cc93781d6abe31bb8f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:45:46.661841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ca1b3-1638-4f48-9282-c14f9f1a0f03 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:46.674432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:46.706081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222e3f6-2aae-438f-bb3e-12b2db2d7344 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:46.752865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff8f0e-ac24-4a49-ae85-b0e9dfa794b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:46.753744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff8f0e-ac24-4a49-ae85-b0e9dfa794b1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:46.754023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff8f0e-ac24-4a49-ae85-b0e9dfa794b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:46.754291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff8f0e-ac24-4a49-ae85-b0e9dfa794b1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:46.755010 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff8f0e-ac24-4a49-ae85-b0e9dfa794b1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:46.757088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff8f0e-ac24-4a49-ae85-b0e9dfa794b1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-15ff8f0e-ac24-4a49-ae85-b0e9dfa794b1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:46.771403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:46.772448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:46.773278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 410/823] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Jun 29 09:45:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:46.774135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff8f0e-ac24-4a49-ae85-b0e9dfa794b1 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:46.804989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ca1b3-1638-4f48-9282-c14f9f1a0f03 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:46.843771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:46.861207 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d61caf-aada-4fc0-bfc5-a236ca7d36fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:46.861889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d61caf-aada-4fc0-bfc5-a236ca7d36fb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:46.862199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d61caf-aada-4fc0-bfc5-a236ca7d36fb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:46.862502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d61caf-aada-4fc0-bfc5-a236ca7d36fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:46.863374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d61caf-aada-4fc0-bfc5-a236ca7d36fb None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=8a5eea7f539048c48ed10e72dd933df9) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:46.863691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d61caf-aada-4fc0-bfc5-a236ca7d36fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:46.865201 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d61caf-aada-4fc0-bfc5-a236ca7d36fb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:46.871918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 414/824] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:46 2022] GET /identity/v3/role_assignments?user.id=8a5eea7f539048c48ed10e72dd933df9&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:46.872635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d61caf-aada-4fc0-bfc5-a236ca7d36fb None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:46.929015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff8f0e-ac24-4a49-ae85-b0e9dfa794b1 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:46.969358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59418b-4850-41ce-8274-a99f89f0ca03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:46.969639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59418b-4850-41ce-8274-a99f89f0ca03 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:46.969953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59418b-4850-41ce-8274-a99f89f0ca03 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:46.970187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59418b-4850-41ce-8274-a99f89f0ca03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:46.983190 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59418b-4850-41ce-8274-a99f89f0ca03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:46.984522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59418b-4850-41ce-8274-a99f89f0ca03 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:46.986830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-8c59418b-4850-41ce-8274-a99f89f0ca03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:47.005483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:47.006459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:47.007388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:47.007388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:47.007388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:47.007388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:47.007388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 411/825] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:47.012503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59418b-4850-41ce-8274-a99f89f0ca03 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:47.021170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d61caf-aada-4fc0-bfc5-a236ca7d36fb None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:47.074731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63adc1-6201-4229-8d92-ae433a6ce5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:47.076165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63adc1-6201-4229-8d92-ae433a6ce5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:47.076954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63adc1-6201-4229-8d92-ae433a6ce5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:47.077209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63adc1-6201-4229-8d92-ae433a6ce5b1 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:47.077969 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63adc1-6201-4229-8d92-ae433a6ce5b1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:47.078506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63adc1-6201-4229-8d92-ae433a6ce5b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:47.081856 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63adc1-6201-4229-8d92-ae433a6ce5b1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:47.088734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 415/826] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 09:45:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:47.097800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63adc1-6201-4229-8d92-ae433a6ce5b1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:47.169970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59418b-4850-41ce-8274-a99f89f0ca03 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:47.170592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:47.207976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9859a-86e0-49c1-a8e5-3e05fd93638d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:47.208846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9859a-86e0-49c1-a8e5-3e05fd93638d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:47.209223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9859a-86e0-49c1-a8e5-3e05fd93638d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:47.209586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9859a-86e0-49c1-a8e5-3e05fd93638d None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:47.210436 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9859a-86e0-49c1-a8e5-3e05fd93638d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:47.210869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9859a-86e0-49c1-a8e5-3e05fd93638d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:47.212341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9859a-86e0-49c1-a8e5-3e05fd93638d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:47.227853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 412/827] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:47.233739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9859a-86e0-49c1-a8e5-3e05fd93638d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:47.245032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63adc1-6201-4229-8d92-ae433a6ce5b1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:47.290103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0dc53-0620-4aa5-90c5-d98c081ee469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:47.290812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0dc53-0620-4aa5-90c5-d98c081ee469 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:47.291086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0dc53-0620-4aa5-90c5-d98c081ee469 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:47.291348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0dc53-0620-4aa5-90c5-d98c081ee469 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:47.292227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0dc53-0620-4aa5-90c5-d98c081ee469 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:47.292536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0dc53-0620-4aa5-90c5-d98c081ee469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:47.293983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0dc53-0620-4aa5-90c5-d98c081ee469 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:47.302601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:47.302796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 416/828] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 09:45:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:47.306257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0dc53-0620-4aa5-90c5-d98c081ee469 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:47.368020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9859a-86e0-49c1-a8e5-3e05fd93638d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:47.416739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33be3f-f52f-445c-8fc2-62a5292f9b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:47.417954 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33be3f-f52f-445c-8fc2-62a5292f9b59 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:47.418338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33be3f-f52f-445c-8fc2-62a5292f9b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:47.418696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33be3f-f52f-445c-8fc2-62a5292f9b59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:47.422374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0dc53-0620-4aa5-90c5-d98c081ee469 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:47.428786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33be3f-f52f-445c-8fc2-62a5292f9b59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:47.430343 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33be3f-f52f-445c-8fc2-62a5292f9b59 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-4f33be3f-f52f-445c-8fc2-62a5292f9b59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:47.447668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:47.448612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:47.449571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:47.449571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:47.449571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:47.449571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:47.449571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:47.449571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:47.449571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:47.449571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:47.450004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:47.452344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 413/829] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:47.462235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33be3f-f52f-445c-8fc2-62a5292f9b59 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:47.476767 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4ddac-3b4a-4d3d-a863-7801106f9f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:47.477489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4ddac-3b4a-4d3d-a863-7801106f9f34 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:47.478107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4ddac-3b4a-4d3d-a863-7801106f9f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:47.478214 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4ddac-3b4a-4d3d-a863-7801106f9f34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:47.478935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4ddac-3b4a-4d3d-a863-7801106f9f34 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:47.480318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4ddac-3b4a-4d3d-a863-7801106f9f34 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:47.510597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 417/830] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 09:45:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:45:47.589286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33be3f-f52f-445c-8fc2-62a5292f9b59 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:47.605509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:47.637016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba11ad-11aa-4335-b997-a5b570bfe4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:47.637696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba11ad-11aa-4335-b997-a5b570bfe4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:47.637972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba11ad-11aa-4335-b997-a5b570bfe4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:47.638239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba11ad-11aa-4335-b997-a5b570bfe4b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:47.639067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba11ad-11aa-4335-b997-a5b570bfe4b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:47.639372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba11ad-11aa-4335-b997-a5b570bfe4b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:47.640840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba11ad-11aa-4335-b997-a5b570bfe4b9 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:47.655310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 414/831] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:47.662771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4ddac-3b4a-4d3d-a863-7801106f9f34 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:47.783948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4ddac-3b4a-4d3d-a863-7801106f9f34 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:47.827351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa5f49-f1b1-403b-9687-689457e613ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:47.828282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa5f49-f1b1-403b-9687-689457e613ff None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:47.828566 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa5f49-f1b1-403b-9687-689457e613ff None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:47.829008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa5f49-f1b1-403b-9687-689457e613ff None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/d86d073ef5ee4631b508ab578662be96/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:47.831310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa5f49-f1b1-403b-9687-689457e613ff None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=d86d073ef5ee4631b508ab578662be96, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:47.832812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa5f49-f1b1-403b-9687-689457e613ff None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:47.849956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 418/832] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:45:47 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/d86d073ef5ee4631b508ab578662be96/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:45:48.781720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da66e6d-f8f9-4a48-88b8-b95b171c529b None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:48.782970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da66e6d-f8f9-4a48-88b8-b95b171c529b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:48.782970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da66e6d-f8f9-4a48-88b8-b95b171c529b None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:48.784095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 415/833] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:48.789771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38248b34-7c4f-4638-868f-e0da61a5afc8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:48.790243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38248b34-7c4f-4638-868f-e0da61a5afc8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:48.790322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38248b34-7c4f-4638-868f-e0da61a5afc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:48.808504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-38248b34-7c4f-4638-868f-e0da61a5afc8 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:48.809714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-38248b34-7c4f-4638-868f-e0da61a5afc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:48.837626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:48.890691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 419/834] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:48.908968 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da66e6d-f8f9-4a48-88b8-b95b171c529b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:49.019069 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da66e6d-f8f9-4a48-88b8-b95b171c529b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:49.046648 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fc83a-05c5-45ee-a682-46c4be996589 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.046899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fc83a-05c5-45ee-a682-46c4be996589 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.047143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fc83a-05c5-45ee-a682-46c4be996589 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.049079 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 420/835] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:49.055482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec691e5-8e88-438a-a1e6-c7a32e77a17e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.055962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec691e5-8e88-438a-a1e6-c7a32e77a17e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.056457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec691e5-8e88-438a-a1e6-c7a32e77a17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.059033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e785ad-9b4b-45fc-a511-d9d45f353830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:49.060255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e785ad-9b4b-45fc-a511-d9d45f353830 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.060725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e785ad-9b4b-45fc-a511-d9d45f353830 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.061184 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e785ad-9b4b-45fc-a511-d9d45f353830 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.073319 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e785ad-9b4b-45fc-a511-d9d45f353830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-82e785ad-9b4b-45fc-a511-d9d45f353830 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:49.082241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:49.083249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:45:49.084125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:49.084772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0ec691e5-8e88-438a-a1e6-c7a32e77a17e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:49.084772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0ec691e5-8e88-438a-a1e6-c7a32e77a17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:49.087286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 416/836] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:45:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:49.096517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e785ad-9b4b-45fc-a511-d9d45f353830 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:49.154952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:49.167876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 421/837] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:49.189900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec691e5-8e88-438a-a1e6-c7a32e77a17e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:49.218604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e785ad-9b4b-45fc-a511-d9d45f353830 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:49.258812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483ed21-c16f-40b1-8fba-9f25bb118d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:49.259997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483ed21-c16f-40b1-8fba-9f25bb118d9e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.260476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483ed21-c16f-40b1-8fba-9f25bb118d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.260974 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483ed21-c16f-40b1-8fba-9f25bb118d9e None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.262087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483ed21-c16f-40b1-8fba-9f25bb118d9e None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:49.262637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483ed21-c16f-40b1-8fba-9f25bb118d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:49.264341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483ed21-c16f-40b1-8fba-9f25bb118d9e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:49.276252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 417/838] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:45:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:49.282961 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483ed21-c16f-40b1-8fba-9f25bb118d9e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:49.307117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:49.316420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec691e5-8e88-438a-a1e6-c7a32e77a17e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:49.352504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b53e1c-35c5-4338-a759-8b37f1f7ebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:49.353345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b53e1c-35c5-4338-a759-8b37f1f7ebfa None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.353682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b53e1c-35c5-4338-a759-8b37f1f7ebfa None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.353974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b53e1c-35c5-4338-a759-8b37f1f7ebfa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.354690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b53e1c-35c5-4338-a759-8b37f1f7ebfa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:49.356116 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b53e1c-35c5-4338-a759-8b37f1f7ebfa None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f1b53e1c-35c5-4338-a759-8b37f1f7ebfa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:49.369252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:49.370293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 09:45:49.371563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:49.376098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 422/839] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Jun 29 09:45:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:49.378008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b53e1c-35c5-4338-a759-8b37f1f7ebfa None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:49.393639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:49.403900 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483ed21-c16f-40b1-8fba-9f25bb118d9e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:49.439994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9116e0e-ec48-46e3-a743-2a946dcf70d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:49.441231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116e0e-ec48-46e3-a743-2a946dcf70d4 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.441713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116e0e-ec48-46e3-a743-2a946dcf70d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.442185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116e0e-ec48-46e3-a743-2a946dcf70d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.455127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9116e0e-ec48-46e3-a743-2a946dcf70d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:49.456994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9116e0e-ec48-46e3-a743-2a946dcf70d4 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-e9116e0e-ec48-46e3-a743-2a946dcf70d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:49.476179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:49.477151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:49.478041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:49.478041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:49.478041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:49.478041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:49.484801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 418/840] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:49.486679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9116e0e-ec48-46e3-a743-2a946dcf70d4 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:49.491448 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b53e1c-35c5-4338-a759-8b37f1f7ebfa None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:49.531965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739f86e-9ac8-4ad3-92a7-62e73dbcbbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:49.532548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739f86e-9ac8-4ad3-92a7-62e73dbcbbae None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.532900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739f86e-9ac8-4ad3-92a7-62e73dbcbbae None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.533202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739f86e-9ac8-4ad3-92a7-62e73dbcbbae None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.533984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739f86e-9ac8-4ad3-92a7-62e73dbcbbae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:49.534298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739f86e-9ac8-4ad3-92a7-62e73dbcbbae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:49.536285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739f86e-9ac8-4ad3-92a7-62e73dbcbbae None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:49.550837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 423/841] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 09:45:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:49.553893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739f86e-9ac8-4ad3-92a7-62e73dbcbbae None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:49.587956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:49.603927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9116e0e-ec48-46e3-a743-2a946dcf70d4 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:49.640846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0940a-2634-4b6a-bff6-81b428e476bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:49.642460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0940a-2634-4b6a-bff6-81b428e476bf None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.643008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0940a-2634-4b6a-bff6-81b428e476bf None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.643496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0940a-2634-4b6a-bff6-81b428e476bf None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.644429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0940a-2634-4b6a-bff6-81b428e476bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:49.644936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0940a-2634-4b6a-bff6-81b428e476bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:49.646517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0940a-2634-4b6a-bff6-81b428e476bf None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:49.664726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 419/842] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:49.666407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0940a-2634-4b6a-bff6-81b428e476bf None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:49.670650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:49.677192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739f86e-9ac8-4ad3-92a7-62e73dbcbbae None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:49.715441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c30c1-d00b-4598-ae83-2c9f0381d972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:49.716721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c30c1-d00b-4598-ae83-2c9f0381d972 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.717412 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c30c1-d00b-4598-ae83-2c9f0381d972 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.717906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c30c1-d00b-4598-ae83-2c9f0381d972 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.718903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c30c1-d00b-4598-ae83-2c9f0381d972 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:49.719462 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c30c1-d00b-4598-ae83-2c9f0381d972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:49.721356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c30c1-d00b-4598-ae83-2c9f0381d972 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:49.734086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 424/843] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 09:45:49 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:49.770015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a0c2e-beee-424f-9424-ef3d3b68eb89 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.770335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a0c2e-beee-424f-9424-ef3d3b68eb89 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.770688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a0c2e-beee-424f-9424-ef3d3b68eb89 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.772349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 425/844] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:49.778079 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b141395-6786-44a6-b3a9-e417764c1f01 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.778401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b141395-6786-44a6-b3a9-e417764c1f01 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.778677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b141395-6786-44a6-b3a9-e417764c1f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.780266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0940a-2634-4b6a-bff6-81b428e476bf None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:49.797794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2b141395-6786-44a6-b3a9-e417764c1f01 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:49.799834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2b141395-6786-44a6-b3a9-e417764c1f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:49.820942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b7fb0-a905-4c56-922f-7c3bed561434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:49.821779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b7fb0-a905-4c56-922f-7c3bed561434 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:49.822057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b7fb0-a905-4c56-922f-7c3bed561434 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:49.822328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b7fb0-a905-4c56-922f-7c3bed561434 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:49.831822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b7fb0-a905-4c56-922f-7c3bed561434 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:49.833401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b7fb0-a905-4c56-922f-7c3bed561434 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-dd0b7fb0-a905-4c56-922f-7c3bed561434 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:49.851614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:49.852580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:49.853510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:49.853510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:49.853510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:49.853510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:49.853510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:49.853510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:49.853510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:49.853510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:49.853510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:45:49.853510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:49.857514 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 420/845] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 09:45:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:49.859447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b7fb0-a905-4c56-922f-7c3bed561434 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:49.895446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 426/846] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:45:49.921412 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b141395-6786-44a6-b3a9-e417764c1f01 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:49.941680 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:49.943979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:49.997990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b7fb0-a905-4c56-922f-7c3bed561434 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:50.035303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b141395-6786-44a6-b3a9-e417764c1f01 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:50.050604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549d350-ff9a-4919-84dd-123a6fca17e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:50.050604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549d350-ff9a-4919-84dd-123a6fca17e0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:50.050604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549d350-ff9a-4919-84dd-123a6fca17e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:50.050604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549d350-ff9a-4919-84dd-123a6fca17e0 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:50.051848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549d350-ff9a-4919-84dd-123a6fca17e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:50.052254 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549d350-ff9a-4919-84dd-123a6fca17e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:50.053946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549d350-ff9a-4919-84dd-123a6fca17e0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:50.077595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 421/847] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 09:45:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:50.084002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549d350-ff9a-4919-84dd-123a6fca17e0 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:50.088994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5111df5-fc51-45d7-a7c4-fd13d359159b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:50.090327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5111df5-fc51-45d7-a7c4-fd13d359159b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:50.090817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5111df5-fc51-45d7-a7c4-fd13d359159b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:50.091267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5111df5-fc51-45d7-a7c4-fd13d359159b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:50.092408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5111df5-fc51-45d7-a7c4-fd13d359159b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:50.094863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5111df5-fc51-45d7-a7c4-fd13d359159b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-a5111df5-fc51-45d7-a7c4-fd13d359159b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:50.117088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:50.118136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 09:45:50.119048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:50.120525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 427/848] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:50.121631 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396bfe2-fc94-41a1-8d29-fb9246d9bfe5 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:50.122103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396bfe2-fc94-41a1-8d29-fb9246d9bfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:50.122537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396bfe2-fc94-41a1-8d29-fb9246d9bfe5 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:50.124837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 428/849] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:50.125996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396bfe2-fc94-41a1-8d29-fb9246d9bfe5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:50.215073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549d350-ff9a-4919-84dd-123a6fca17e0 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:50.232126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:50.243405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:50.257427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7ae78-1106-41b5-ab6e-d93ef5dd015e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:50.258229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7ae78-1106-41b5-ab6e-d93ef5dd015e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:50.258514 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7ae78-1106-41b5-ab6e-d93ef5dd015e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:50.258818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7ae78-1106-41b5-ab6e-d93ef5dd015e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:50.259410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396bfe2-fc94-41a1-8d29-fb9246d9bfe5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:50.297987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50ea94-752c-48d9-8939-b20c18018fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:50.298860 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50ea94-752c-48d9-8939-b20c18018fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:50.299141 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50ea94-752c-48d9-8939-b20c18018fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:50.299413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50ea94-752c-48d9-8939-b20c18018fc6 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:50.300128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50ea94-752c-48d9-8939-b20c18018fc6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:50.300435 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50ea94-752c-48d9-8939-b20c18018fc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:50.302003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50ea94-752c-48d9-8939-b20c18018fc6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:50.307658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 429/850] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:50.309039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45071-ab35-43d2-95ae-ed09c76482a7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:50.309330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45071-ab35-43d2-95ae-ed09c76482a7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:50.309608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45071-ab35-43d2-95ae-ed09c76482a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:50.327543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e0b45071-ab35-43d2-95ae-ed09c76482a7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:50.328726 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e0b45071-ab35-43d2-95ae-ed09c76482a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:50.344974 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7ae78-1106-41b5-ab6e-d93ef5dd015e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:50.346503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7ae78-1106-41b5-ab6e-d93ef5dd015e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:50.403741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 430/851] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:50.404582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b45071-ab35-43d2-95ae-ed09c76482a7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:50.460800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 422/852] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:50 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 09:45:50.461957 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7ae78-1106-41b5-ab6e-d93ef5dd015e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:50.488168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:50.506652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:50.538629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b45071-ab35-43d2-95ae-ed09c76482a7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:50.572664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7ae78-1106-41b5-ab6e-d93ef5dd015e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:50.576022 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80878c3c-0a4f-4967-ac42-e73e518a7c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:50.576705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80878c3c-0a4f-4967-ac42-e73e518a7c59 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:50.576991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80878c3c-0a4f-4967-ac42-e73e518a7c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:50.577271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80878c3c-0a4f-4967-ac42-e73e518a7c59 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:50.577987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80878c3c-0a4f-4967-ac42-e73e518a7c59 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:50.578297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80878c3c-0a4f-4967-ac42-e73e518a7c59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:50.579765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80878c3c-0a4f-4967-ac42-e73e518a7c59 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:50.586159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 431/853] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:50 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:50.586890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80878c3c-0a4f-4967-ac42-e73e518a7c59 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:50.620662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b647fa2-40ba-4645-b19d-415006e8637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:50.621996 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b647fa2-40ba-4645-b19d-415006e8637a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:50.622482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b647fa2-40ba-4645-b19d-415006e8637a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:50.623822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b647fa2-40ba-4645-b19d-415006e8637a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:50.638945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b647fa2-40ba-4645-b19d-415006e8637a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-3b647fa2-40ba-4645-b19d-415006e8637a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:50.659282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:50.660277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:45:50.661112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:50.664329 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 423/854] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:50.671952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b647fa2-40ba-4645-b19d-415006e8637a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:50.816616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80878c3c-0a4f-4967-ac42-e73e518a7c59 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:50.868414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:50.878459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab928e-c0bf-4727-8d37-ed15417982c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:50.879831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab928e-c0bf-4727-8d37-ed15417982c4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:50.880290 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab928e-c0bf-4727-8d37-ed15417982c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:50.880719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab928e-c0bf-4727-8d37-ed15417982c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:50.886050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:50.900176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab928e-c0bf-4727-8d37-ed15417982c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:50.902316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab928e-c0bf-4727-8d37-ed15417982c4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:50.913716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b647fa2-40ba-4645-b19d-415006e8637a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-1aab928e-c0bf-4727-8d37-ed15417982c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:50.927695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:50.928669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:50.929600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:50.929600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:50.929600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:50.929600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:50.932098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 432/855] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:50.937388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab928e-c0bf-4727-8d37-ed15417982c4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:50.974354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9e0d3-ddb6-4703-9798-d6b904b4ff4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:50.976873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9e0d3-ddb6-4703-9798-d6b904b4ff4b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:50.976873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9e0d3-ddb6-4703-9798-d6b904b4ff4b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:50.977074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9e0d3-ddb6-4703-9798-d6b904b4ff4b None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:50.977695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9e0d3-ddb6-4703-9798-d6b904b4ff4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:50.978005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9e0d3-ddb6-4703-9798-d6b904b4ff4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:50.979853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9e0d3-ddb6-4703-9798-d6b904b4ff4b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:50.996546 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 424/856] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:45:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:50.999471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9e0d3-ddb6-4703-9798-d6b904b4ff4b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:51.131690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab928e-c0bf-4727-8d37-ed15417982c4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:51.180137 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaebf6e-16b6-4bb7-93ec-aa0848496cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:51.181585 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaebf6e-16b6-4bb7-93ec-aa0848496cbc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:51.182295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaebf6e-16b6-4bb7-93ec-aa0848496cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:51.182749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaebf6e-16b6-4bb7-93ec-aa0848496cbc None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:51.183877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaebf6e-16b6-4bb7-93ec-aa0848496cbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:51.184331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaebf6e-16b6-4bb7-93ec-aa0848496cbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:51.186533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaebf6e-16b6-4bb7-93ec-aa0848496cbc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:51.193849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9e0d3-ddb6-4703-9798-d6b904b4ff4b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:51.204240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 433/857] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:51.208945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaebf6e-16b6-4bb7-93ec-aa0848496cbc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:51.233530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484141-f6a9-46bd-9642-951d875f7229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:51.234590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484141-f6a9-46bd-9642-951d875f7229 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:51.235034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484141-f6a9-46bd-9642-951d875f7229 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:51.235413 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484141-f6a9-46bd-9642-951d875f7229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:51.248176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484141-f6a9-46bd-9642-951d875f7229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:51.249790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484141-f6a9-46bd-9642-951d875f7229 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:51.253482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:51.274098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-df484141-f6a9-46bd-9642-951d875f7229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:51.278746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:51.279901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:51.281002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:51.281002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:51.281002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:51.281002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:51.288270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 425/858] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:51.290015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484141-f6a9-46bd-9642-951d875f7229 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:51.341796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaebf6e-16b6-4bb7-93ec-aa0848496cbc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:51.379609 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3e802-0473-47a9-bf13-f02a7fec1478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:51.380437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3e802-0473-47a9-bf13-f02a7fec1478 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:51.380722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3e802-0473-47a9-bf13-f02a7fec1478 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:51.381000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3e802-0473-47a9-bf13-f02a7fec1478 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:51.389291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3e802-0473-47a9-bf13-f02a7fec1478 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:51.390514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3e802-0473-47a9-bf13-f02a7fec1478 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-29c3e802-0473-47a9-bf13-f02a7fec1478 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:51.406469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:51.407814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:51.408812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:51.408812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:51.408812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:51.408812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:51.408812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:51.408812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:51.408812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:51.408812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:51.412854 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 434/859] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:51.413890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3e802-0473-47a9-bf13-f02a7fec1478 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:51.425979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484141-f6a9-46bd-9642-951d875f7229 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:51.467539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f84aa4-b433-4aea-a816-d351b909afe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:51.468802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f84aa4-b433-4aea-a816-d351b909afe3 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:51.469281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f84aa4-b433-4aea-a816-d351b909afe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:51.469787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f84aa4-b433-4aea-a816-d351b909afe3 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:51.471380 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f84aa4-b433-4aea-a816-d351b909afe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:51.472007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f84aa4-b433-4aea-a816-d351b909afe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:51.474192 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f84aa4-b433-4aea-a816-d351b909afe3 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:51.496068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 426/860] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:51.497215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f84aa4-b433-4aea-a816-d351b909afe3 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:51.541362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3e802-0473-47a9-bf13-f02a7fec1478 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:51.557475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:51.558550 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:51.596295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7267622-b162-4cab-931a-8aa0cc18358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:51.597331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7267622-b162-4cab-931a-8aa0cc18358f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:51.597625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7267622-b162-4cab-931a-8aa0cc18358f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:51.597898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7267622-b162-4cab-931a-8aa0cc18358f None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:51.599083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7267622-b162-4cab-931a-8aa0cc18358f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:51.599393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7267622-b162-4cab-931a-8aa0cc18358f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:51.601386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7267622-b162-4cab-931a-8aa0cc18358f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:51.613893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f84aa4-b433-4aea-a816-d351b909afe3 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:51.618507 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 435/861] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:51.625918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7267622-b162-4cab-931a-8aa0cc18358f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:51.667019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e2ba4-7932-48d6-8071-6fc5148cc129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:51.667920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e2ba4-7932-48d6-8071-6fc5148cc129 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:51.668261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e2ba4-7932-48d6-8071-6fc5148cc129 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:51.668593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e2ba4-7932-48d6-8071-6fc5148cc129 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:51.679579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e2ba4-7932-48d6-8071-6fc5148cc129 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:51.681493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e2ba4-7932-48d6-8071-6fc5148cc129 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-e44e2ba4-7932-48d6-8071-6fc5148cc129 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:51.702213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:51.703553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:51.705681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:51.705681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:51.705681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:51.705681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:51.705681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:51.705681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:51.705681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:51.705681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:51.705681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 09:45:51.705681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:51.706399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 427/862] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:45:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:51.712323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e2ba4-7932-48d6-8071-6fc5148cc129 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:51.738546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7267622-b162-4cab-931a-8aa0cc18358f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:51.773981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccd4e6-2ead-4b0e-8d4f-13355e7efa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:51.774829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccd4e6-2ead-4b0e-8d4f-13355e7efa5f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:51.775179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccd4e6-2ead-4b0e-8d4f-13355e7efa5f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:51.775522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccd4e6-2ead-4b0e-8d4f-13355e7efa5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:51.776472 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccd4e6-2ead-4b0e-8d4f-13355e7efa5f None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=a9ac6a36e5224f38bdc5b214580ff18a) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:51.776880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccd4e6-2ead-4b0e-8d4f-13355e7efa5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:51.778456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccd4e6-2ead-4b0e-8d4f-13355e7efa5f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:51.788322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 436/863] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:51 2022] GET /identity/v3/role_assignments?user.id=a9ac6a36e5224f38bdc5b214580ff18a&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:51.844677 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e2ba4-7932-48d6-8071-6fc5148cc129 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:51.865575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:51.884146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77bf60-2651-4a18-9b65-971fa2d321e8 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:51.884679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77bf60-2651-4a18-9b65-971fa2d321e8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:51.884679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77bf60-2651-4a18-9b65-971fa2d321e8 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:51.886405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 437/864] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:45:51.891196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238da13-17e7-4dc2-9b78-d2ee25def862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:51.892013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bc744-46af-42b5-a4a3-9a76aa8555f8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:51.892115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238da13-17e7-4dc2-9b78-d2ee25def862 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:51.892206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bc744-46af-42b5-a4a3-9a76aa8555f8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:51.892295 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238da13-17e7-4dc2-9b78-d2ee25def862 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:51.892535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238da13-17e7-4dc2-9b78-d2ee25def862 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:51.892692 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bc744-46af-42b5-a4a3-9a76aa8555f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:51.893399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238da13-17e7-4dc2-9b78-d2ee25def862 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:51.893713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238da13-17e7-4dc2-9b78-d2ee25def862 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:51.895202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238da13-17e7-4dc2-9b78-d2ee25def862 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:51.910661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-139bc744-46af-42b5-a4a3-9a76aa8555f8 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:51.911965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-139bc744-46af-42b5-a4a3-9a76aa8555f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:51.915658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 428/865] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 09:45:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:51.926685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238da13-17e7-4dc2-9b78-d2ee25def862 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:51.946111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:52.001588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 438/866] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:45:52.024588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bc744-46af-42b5-a4a3-9a76aa8555f8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:52.057320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238da13-17e7-4dc2-9b78-d2ee25def862 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:52.098470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358aa15-9bba-4f3e-8da4-495d359b9cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:52.099523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358aa15-9bba-4f3e-8da4-495d359b9cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:52.099807 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358aa15-9bba-4f3e-8da4-495d359b9cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:52.100080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358aa15-9bba-4f3e-8da4-495d359b9cc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:52.156809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:52.157766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bc744-46af-42b5-a4a3-9a76aa8555f8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:52.193349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358aa15-9bba-4f3e-8da4-495d359b9cc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:52.194865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358aa15-9bba-4f3e-8da4-495d359b9cc8 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:52.199517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa125b34-ac4e-40e3-b0fb-8768d1315575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:52.200261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa125b34-ac4e-40e3-b0fb-8768d1315575 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:52.200547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa125b34-ac4e-40e3-b0fb-8768d1315575 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:52.200823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa125b34-ac4e-40e3-b0fb-8768d1315575 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:52.201623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa125b34-ac4e-40e3-b0fb-8768d1315575 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:52.203520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa125b34-ac4e-40e3-b0fb-8768d1315575 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:52.224021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 439/867] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:45:52 2022] POST /identity/v3/services => generated 269 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:45:52.332274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 429/868] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:51 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:45:52.337728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa125b34-ac4e-40e3-b0fb-8768d1315575 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:52.372056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:52.467707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa125b34-ac4e-40e3-b0fb-8768d1315575 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:52.506622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d9ff2-d2ff-4ad1-96a7-e7341fbe6534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:52.507465 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d9ff2-d2ff-4ad1-96a7-e7341fbe6534 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:52.507748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d9ff2-d2ff-4ad1-96a7-e7341fbe6534 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:52.508023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d9ff2-d2ff-4ad1-96a7-e7341fbe6534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:52.520212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d9ff2-d2ff-4ad1-96a7-e7341fbe6534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:52.521702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d9ff2-d2ff-4ad1-96a7-e7341fbe6534 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-ef2d9ff2-d2ff-4ad1-96a7-e7341fbe6534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:52.540536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:52.541506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:52.542401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:52.542401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:52.542401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:52.542401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:52.542401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 440/869] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:52.547494 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358aa15-9bba-4f3e-8da4-495d359b9cc8 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:52.629982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:52.658524 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358aa15-9bba-4f3e-8da4-495d359b9cc8 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:52.695635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33703249-2e02-4a85-aac4-ca6d314145f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:52.696304 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33703249-2e02-4a85-aac4-ca6d314145f7 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:52.696578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33703249-2e02-4a85-aac4-ca6d314145f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:52.696846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33703249-2e02-4a85-aac4-ca6d314145f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:52.697578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33703249-2e02-4a85-aac4-ca6d314145f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:52.697878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33703249-2e02-4a85-aac4-ca6d314145f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:52.699274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33703249-2e02-4a85-aac4-ca6d314145f7 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:52.711967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 430/870] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:52.717526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d9ff2-d2ff-4ad1-96a7-e7341fbe6534 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:52.736745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa5961-1a84-4bc4-a7ea-430c7d543033 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:52.737110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa5961-1a84-4bc4-a7ea-430c7d543033 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:52.737306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa5961-1a84-4bc4-a7ea-430c7d543033 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:52.738988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 431/871] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:52.746441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c90a60-b0e8-44ad-b85c-f778995f46af None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:52.746800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c90a60-b0e8-44ad-b85c-f778995f46af None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:52.747002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c90a60-b0e8-44ad-b85c-f778995f46af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:52.769667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-e5c90a60-b0e8-44ad-b85c-f778995f46af None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:52.770887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e5c90a60-b0e8-44ad-b85c-f778995f46af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:52.803316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:52.836209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d9ff2-d2ff-4ad1-96a7-e7341fbe6534 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:52.850611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 432/872] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:52 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:45:52.873576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c90a60-b0e8-44ad-b85c-f778995f46af None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:52.885703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0f585-bb34-4b2c-839b-9e768c0a7924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:52.887173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0f585-bb34-4b2c-839b-9e768c0a7924 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:52.887674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0f585-bb34-4b2c-839b-9e768c0a7924 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:52.888141 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0f585-bb34-4b2c-839b-9e768c0a7924 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:52.895669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0f585-bb34-4b2c-839b-9e768c0a7924 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:52.897386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0f585-bb34-4b2c-839b-9e768c0a7924 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-abd0f585-bb34-4b2c-839b-9e768c0a7924 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:52.912802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:52.913739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:52.914653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:52.914653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:52.914653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:52.914653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:52.914653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:52.914653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:52.914653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:52.914653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:52.917223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 441/873] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:52.923228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0f585-bb34-4b2c-839b-9e768c0a7924 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:52.951298 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:52.998791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c90a60-b0e8-44ad-b85c-f778995f46af None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:53.035489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f91f-58df-4550-81c0-17bbe7d4eb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:53.036171 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f91f-58df-4550-81c0-17bbe7d4eb7b None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:53.036445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f91f-58df-4550-81c0-17bbe7d4eb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:53.036714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f91f-58df-4550-81c0-17bbe7d4eb7b None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:53.037157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0f585-bb34-4b2c-839b-9e768c0a7924 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:53.037454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26f91f-58df-4550-81c0-17bbe7d4eb7b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:53.044558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26f91f-58df-4550-81c0-17bbe7d4eb7b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:53.063869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 433/874] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:45:52 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:45:53.077385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:53.090455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684659f-9d5b-4e13-9d2e-e14dfbd34fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:53.091288 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684659f-9d5b-4e13-9d2e-e14dfbd34fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:53.091643 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684659f-9d5b-4e13-9d2e-e14dfbd34fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:53.091929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684659f-9d5b-4e13-9d2e-e14dfbd34fd7 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:53.092762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684659f-9d5b-4e13-9d2e-e14dfbd34fd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:53.093083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684659f-9d5b-4e13-9d2e-e14dfbd34fd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:53.094699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684659f-9d5b-4e13-9d2e-e14dfbd34fd7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:53.110630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 442/875] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:53.117119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f91f-58df-4550-81c0-17bbe7d4eb7b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:53.236456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f91f-58df-4550-81c0-17bbe7d4eb7b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:53.261087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:53.274390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ef812-b254-4f47-a864-046c2f9f17f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:53.275074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ef812-b254-4f47-a864-046c2f9f17f7 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:53.276130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ef812-b254-4f47-a864-046c2f9f17f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:53.276130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ef812-b254-4f47-a864-046c2f9f17f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:53.276495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ef812-b254-4f47-a864-046c2f9f17f7 None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=d86d073ef5ee4631b508ab578662be96) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:53.276802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ef812-b254-4f47-a864-046c2f9f17f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:53.278255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ef812-b254-4f47-a864-046c2f9f17f7 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:53.285534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 434/876] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:53 2022] GET /identity/v3/role_assignments?user.id=d86d073ef5ee4631b508ab578662be96&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:54.055710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ffa9b-81b9-4013-b74a-2d0c1088c791 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:54.056165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ffa9b-81b9-4013-b74a-2d0c1088c791 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:54.056250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ffa9b-81b9-4013-b74a-2d0c1088c791 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:54.057903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 443/877] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:54.063305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e600484-99bc-4769-be95-207efec70b86 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:54.063735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e600484-99bc-4769-be95-207efec70b86 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:54.063905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e600484-99bc-4769-be95-207efec70b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:54.081287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-9e600484-99bc-4769-be95-207efec70b86 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:54.082501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9e600484-99bc-4769-be95-207efec70b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:54.153359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 435/878] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:54.177096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ffa9b-81b9-4013-b74a-2d0c1088c791 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:54.287483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ffa9b-81b9-4013-b74a-2d0c1088c791 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:54.323356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d14f11-2bb5-4be4-83a5-40b7b28462c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:54.324092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d14f11-2bb5-4be4-83a5-40b7b28462c1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:54.324374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d14f11-2bb5-4be4-83a5-40b7b28462c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:54.324651 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d14f11-2bb5-4be4-83a5-40b7b28462c1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:54.335767 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d14f11-2bb5-4be4-83a5-40b7b28462c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-20d14f11-2bb5-4be4-83a5-40b7b28462c1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:54.344001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:54.345513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:54.347010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 444/879] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 09:45:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:54.352076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e600484-99bc-4769-be95-207efec70b86 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:54.401236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77384499-0cb3-49bd-9e34-b05485d3eac7 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:54.401541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77384499-0cb3-49bd-9e34-b05485d3eac7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:54.401789 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77384499-0cb3-49bd-9e34-b05485d3eac7 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:54.403471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 445/880] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:54.408724 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a817bc9-de99-480c-82e7-ab0dede05e38 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:54.409010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a817bc9-de99-480c-82e7-ab0dede05e38 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:54.409287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a817bc9-de99-480c-82e7-ab0dede05e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:54.427768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6a817bc9-de99-480c-82e7-ab0dede05e38 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:54.429050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6a817bc9-de99-480c-82e7-ab0dede05e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:54.464446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e600484-99bc-4769-be95-207efec70b86 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:54.479086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:54.483498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:54.501696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae607f9c-6842-4f31-8f5e-ea9bbc1b886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:54.502422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae607f9c-6842-4f31-8f5e-ea9bbc1b886f None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:54.502734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae607f9c-6842-4f31-8f5e-ea9bbc1b886f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:54.503055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae607f9c-6842-4f31-8f5e-ea9bbc1b886f None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:54.503809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae607f9c-6842-4f31-8f5e-ea9bbc1b886f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:54.504112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae607f9c-6842-4f31-8f5e-ea9bbc1b886f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:54.505527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae607f9c-6842-4f31-8f5e-ea9bbc1b886f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:54.514864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 436/882] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 09:45:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:54.515120 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 446/882] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:45:54.520202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae607f9c-6842-4f31-8f5e-ea9bbc1b886f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:54.533143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a817bc9-de99-480c-82e7-ab0dede05e38 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:54.623727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae607f9c-6842-4f31-8f5e-ea9bbc1b886f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:54.640787 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a817bc9-de99-480c-82e7-ab0dede05e38 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:54.659097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0152f-7b82-42c9-8dd4-d475626cd8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:54.659899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0152f-7b82-42c9-8dd4-d475626cd8ef None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:54.660175 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0152f-7b82-42c9-8dd4-d475626cd8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:54.660444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0152f-7b82-42c9-8dd4-d475626cd8ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:54.671437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0152f-7b82-42c9-8dd4-d475626cd8ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:54.672965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0152f-7b82-42c9-8dd4-d475626cd8ef None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:54.677698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05e8dc-b744-4771-aef0-4af14cdb3447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:54.678526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05e8dc-b744-4771-aef0-4af14cdb3447 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:54.678834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05e8dc-b744-4771-aef0-4af14cdb3447 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:54.679106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05e8dc-b744-4771-aef0-4af14cdb3447 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:54.679882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05e8dc-b744-4771-aef0-4af14cdb3447 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:54.681354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05e8dc-b744-4771-aef0-4af14cdb3447 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-64b0152f-7b82-42c9-8dd4-d475626cd8ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:54.690417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:54.692530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:54.694422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:54.694422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:54.694422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:54.694422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:54.694422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 437/883] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-de05e8dc-b744-4771-aef0-4af14cdb3447 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:54.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:54.696865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 09:45:54.698537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:54.702769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0152f-7b82-42c9-8dd4-d475626cd8ef None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:54.702887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 447/884] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Jun 29 09:45:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:54.702887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05e8dc-b744-4771-aef0-4af14cdb3447 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:54.732693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:54.757243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:54.807587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0152f-7b82-42c9-8dd4-d475626cd8ef None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:54.822964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05e8dc-b744-4771-aef0-4af14cdb3447 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:54.845311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4eaa85-aadd-4efd-b4d9-1025f83910f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:54.845750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4eaa85-aadd-4efd-b4d9-1025f83910f3 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:54.846032 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4eaa85-aadd-4efd-b4d9-1025f83910f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:54.846299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4eaa85-aadd-4efd-b4d9-1025f83910f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:54.847053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4eaa85-aadd-4efd-b4d9-1025f83910f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:54.847355 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4eaa85-aadd-4efd-b4d9-1025f83910f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:54.849205 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4eaa85-aadd-4efd-b4d9-1025f83910f3 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:54.863861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 438/885] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:54.865571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd70d38-50fa-41da-9e62-1fe9e311c288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:54.866675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd70d38-50fa-41da-9e62-1fe9e311c288 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:54.867168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd70d38-50fa-41da-9e62-1fe9e311c288 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:54.867633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd70d38-50fa-41da-9e62-1fe9e311c288 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:54.868586 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd70d38-50fa-41da-9e62-1fe9e311c288 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:54.868892 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd70d38-50fa-41da-9e62-1fe9e311c288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:54.870324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd70d38-50fa-41da-9e62-1fe9e311c288 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:54.871666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4eaa85-aadd-4efd-b4d9-1025f83910f3 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:54.881872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 448/886] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 09:45:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:45:54.882811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd70d38-50fa-41da-9e62-1fe9e311c288 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:54.987075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:55.000894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4eaa85-aadd-4efd-b4d9-1025f83910f3 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:55.008494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd70d38-50fa-41da-9e62-1fe9e311c288 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:55.012318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:55.041885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1bc71-9ed8-413f-92ed-74cd14c6c611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:55.042841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1bc71-9ed8-413f-92ed-74cd14c6c611 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.043126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1bc71-9ed8-413f-92ed-74cd14c6c611 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.043403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1bc71-9ed8-413f-92ed-74cd14c6c611 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.052064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1bc71-9ed8-413f-92ed-74cd14c6c611 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:55.053540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1bc71-9ed8-413f-92ed-74cd14c6c611 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:55.058855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494097f-1edf-4503-a727-5924837ae0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:55.060006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494097f-1edf-4503-a727-5924837ae0ac None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.060414 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494097f-1edf-4503-a727-5924837ae0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.060823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494097f-1edf-4503-a727-5924837ae0ac None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.061680 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494097f-1edf-4503-a727-5924837ae0ac None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:55.062122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494097f-1edf-4503-a727-5924837ae0ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:55.064689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494097f-1edf-4503-a727-5924837ae0ac None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-51a1bc71-9ed8-413f-92ed-74cd14c6c611 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:45:55.075059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:55.076348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:55.077275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:55.077275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:55.077275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:45:55.077275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:45:55.077275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:45:55.077275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:45:55.077275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:45:55.077275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:45:55.077275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:45:55.077275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:55.078265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 439/888] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 09:45:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:55.078458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 449/888] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 09:45:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:55.078899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1bc71-9ed8-413f-92ed-74cd14c6c611 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:55.080475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494097f-1edf-4503-a727-5924837ae0ac None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:55.201960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1bc71-9ed8-413f-92ed-74cd14c6c611 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:55.230143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494097f-1edf-4503-a727-5924837ae0ac None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:55.241982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a2869-b44d-4159-8155-ab0ddae50dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:55.243025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a2869-b44d-4159-8155-ab0ddae50dcc None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.243501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a2869-b44d-4159-8155-ab0ddae50dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.243963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a2869-b44d-4159-8155-ab0ddae50dcc None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.245002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a2869-b44d-4159-8155-ab0ddae50dcc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:55.245509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a2869-b44d-4159-8155-ab0ddae50dcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:55.247002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a2869-b44d-4159-8155-ab0ddae50dcc None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:55.265792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 440/889] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 09:45:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:55.268274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d78788-a583-4f5d-80cd-7781b36f56c5 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.268558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d78788-a583-4f5d-80cd-7781b36f56c5 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.268838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d78788-a583-4f5d-80cd-7781b36f56c5 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.270273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 441/890] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:55.271725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d78788-a583-4f5d-80cd-7781b36f56c5 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:55.277390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37463a9d-493b-43de-97b7-859b900674c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:55.278061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37463a9d-493b-43de-97b7-859b900674c4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.278339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37463a9d-493b-43de-97b7-859b900674c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.278639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37463a9d-493b-43de-97b7-859b900674c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.279438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37463a9d-493b-43de-97b7-859b900674c4 None admin] RBAC: Adding query filter params (interface=public, service_id=4cafe753727047c79ac00e9c368e6146, region_id=RegionOne) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:55.279749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37463a9d-493b-43de-97b7-859b900674c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:55.281136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37463a9d-493b-43de-97b7-859b900674c4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:55.285532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:55.290092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 450/891] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 09:45:55 2022] GET /identity/v3/endpoints?service_id=4cafe753727047c79ac00e9c368e6146&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:55.291319 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b23d4-2978-4124-a9ff-bc49e5b0739e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.291608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b23d4-2978-4124-a9ff-bc49e5b0739e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.291880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b23d4-2978-4124-a9ff-bc49e5b0739e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.309881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8b2b23d4-2978-4124-a9ff-bc49e5b0739e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:55.311095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8b2b23d4-2978-4124-a9ff-bc49e5b0739e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:55.338972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:55.390628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 451/892] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:55.394672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b23d4-2978-4124-a9ff-bc49e5b0739e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:55.401884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d78788-a583-4f5d-80cd-7781b36f56c5 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:55.445218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41a185-c39c-4f0e-b410-3de2fb1ba6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:55.445876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41a185-c39c-4f0e-b410-3de2fb1ba6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.446154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41a185-c39c-4f0e-b410-3de2fb1ba6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.446426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41a185-c39c-4f0e-b410-3de2fb1ba6a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.497572 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b23d4-2978-4124-a9ff-bc49e5b0739e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:55.512960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41a185-c39c-4f0e-b410-3de2fb1ba6a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:55.514421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41a185-c39c-4f0e-b410-3de2fb1ba6a7 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:55.527951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:55.534153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e3239-d3a2-447f-909f-dbc7e0ebae8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:55.534883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e3239-d3a2-447f-909f-dbc7e0ebae8f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.535167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e3239-d3a2-447f-909f-dbc7e0ebae8f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.535442 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e3239-d3a2-447f-909f-dbc7e0ebae8f None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.536147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e3239-d3a2-447f-909f-dbc7e0ebae8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:55.537542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e3239-d3a2-447f-909f-dbc7e0ebae8f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:55.543760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 452/893] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 09:45:55 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:45:55.545072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9963f6-88af-4ed9-85aa-337449d6900c None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.545362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9963f6-88af-4ed9-85aa-337449d6900c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.545639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9963f6-88af-4ed9-85aa-337449d6900c None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.551085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 453/894] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:55.551085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9963f6-88af-4ed9-85aa-337449d6900c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:55.584900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:55.646638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 442/895] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:45:55 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 09:45:55.648138 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac6f4e-73ec-4b87-a196-61f30187de35 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.648482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac6f4e-73ec-4b87-a196-61f30187de35 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.648820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac6f4e-73ec-4b87-a196-61f30187de35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.669127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-bbac6f4e-73ec-4b87-a196-61f30187de35 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:55.670603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-bbac6f4e-73ec-4b87-a196-61f30187de35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:55.684397 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9963f6-88af-4ed9-85aa-337449d6900c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:55.733932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e45efc-812d-4f95-b788-55aea59bf91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:55.734795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e45efc-812d-4f95-b788-55aea59bf91c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.735191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e45efc-812d-4f95-b788-55aea59bf91c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.735479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e45efc-812d-4f95-b788-55aea59bf91c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.736157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e45efc-812d-4f95-b788-55aea59bf91c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:55.738809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e45efc-812d-4f95-b788-55aea59bf91c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:55.749056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 443/896] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:55.749963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac6f4e-73ec-4b87-a196-61f30187de35 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-75e45efc-812d-4f95-b788-55aea59bf91c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:55.750080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:55.751894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:55.753043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 454/897] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:55.757529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e45efc-812d-4f95-b788-55aea59bf91c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:55.795422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:55.856680 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:55.865084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac6f4e-73ec-4b87-a196-61f30187de35 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:55.869973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e45efc-812d-4f95-b788-55aea59bf91c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:55.904824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28923f58-c2f7-417a-b21d-b774391bcc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:55.906011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28923f58-c2f7-417a-b21d-b774391bcc69 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.906492 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28923f58-c2f7-417a-b21d-b774391bcc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.907169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28923f58-c2f7-417a-b21d-b774391bcc69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.909451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2adfb-a4c5-4697-8507-1bf96b48acea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:55.910110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2adfb-a4c5-4697-8507-1bf96b48acea None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:55.910389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2adfb-a4c5-4697-8507-1bf96b48acea None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:55.910692 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2adfb-a4c5-4697-8507-1bf96b48acea None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:55.911420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2adfb-a4c5-4697-8507-1bf96b48acea None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:55.911700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2adfb-a4c5-4697-8507-1bf96b48acea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:55.913081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2adfb-a4c5-4697-8507-1bf96b48acea None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:55.918383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 455/898] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:55.919019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2adfb-a4c5-4697-8507-1bf96b48acea None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:55.920867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28923f58-c2f7-417a-b21d-b774391bcc69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:55.922427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28923f58-c2f7-417a-b21d-b774391bcc69 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-28923f58-c2f7-417a-b21d-b774391bcc69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:55.942672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:55.943682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:55.944579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:55.944579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:55.944579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:55.944579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:55.944579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 444/899] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:55.945293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28923f58-c2f7-417a-b21d-b774391bcc69 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:56.031938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2adfb-a4c5-4697-8507-1bf96b48acea None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:56.047106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:56.061880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28923f58-c2f7-417a-b21d-b774391bcc69 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:56.075655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d92859-7e06-4b6d-9df2-2d5f4ece4023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:56.076596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d92859-7e06-4b6d-9df2-2d5f4ece4023 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:56.076878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d92859-7e06-4b6d-9df2-2d5f4ece4023 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:56.077268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d92859-7e06-4b6d-9df2-2d5f4ece4023 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:56.077927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d92859-7e06-4b6d-9df2-2d5f4ece4023 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:56.079654 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d92859-7e06-4b6d-9df2-2d5f4ece4023 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-e6d92859-7e06-4b6d-9df2-2d5f4ece4023 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:56.092048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:56.093112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:56.093976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 456/900] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 09:45:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:56.094984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d92859-7e06-4b6d-9df2-2d5f4ece4023 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:56.105199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fd7bb-6431-43e7-aecd-126794f2f1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:56.106140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fd7bb-6431-43e7-aecd-126794f2f1ac None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:56.106273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fd7bb-6431-43e7-aecd-126794f2f1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:56.106509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fd7bb-6431-43e7-aecd-126794f2f1ac None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:56.107252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fd7bb-6431-43e7-aecd-126794f2f1ac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:56.107555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fd7bb-6431-43e7-aecd-126794f2f1ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:56.109062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fd7bb-6431-43e7-aecd-126794f2f1ac None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:56.114713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 445/901] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:56.115518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fd7bb-6431-43e7-aecd-126794f2f1ac None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:56.126306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:56.215811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d92859-7e06-4b6d-9df2-2d5f4ece4023 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:56.226028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fd7bb-6431-43e7-aecd-126794f2f1ac None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:56.255106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2585c62-db36-4235-a537-804726f5aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:56.256185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2585c62-db36-4235-a537-804726f5aeaa None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:56.256695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2585c62-db36-4235-a537-804726f5aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:56.257040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2585c62-db36-4235-a537-804726f5aeaa None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:56.257777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2585c62-db36-4235-a537-804726f5aeaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:56.258129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2585c62-db36-4235-a537-804726f5aeaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:56.259862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2585c62-db36-4235-a537-804726f5aeaa None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:56.261525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fbb59-bc54-460c-b2c5-8c4b648d6cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:56.262209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fbb59-bc54-460c-b2c5-8c4b648d6cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:56.262820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fbb59-bc54-460c-b2c5-8c4b648d6cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:56.262820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fbb59-bc54-460c-b2c5-8c4b648d6cf9 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:56.263450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fbb59-bc54-460c-b2c5-8c4b648d6cf9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:56.263749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fbb59-bc54-460c-b2c5-8c4b648d6cf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:56.265092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fbb59-bc54-460c-b2c5-8c4b648d6cf9 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:56.274537 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 446/902] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 09:45:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:56.274537 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fbb59-bc54-460c-b2c5-8c4b648d6cf9 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:56.281823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 457/903] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:56.281823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2585c62-db36-4235-a537-804726f5aeaa None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:56.296738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:56.367278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:56.387889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fbb59-bc54-460c-b2c5-8c4b648d6cf9 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:56.392451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2585c62-db36-4235-a537-804726f5aeaa None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:56.427228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39cfa2-ffc5-4095-ba81-16133979775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:56.427896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39cfa2-ffc5-4095-ba81-16133979775e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:56.428179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39cfa2-ffc5-4095-ba81-16133979775e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:56.428452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39cfa2-ffc5-4095-ba81-16133979775e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:56.429348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39cfa2-ffc5-4095-ba81-16133979775e None admin] RBAC: Adding query filter params (interface=public, service_id=8219d0daff7e4f5787b1ea57c17aa9d8, region_id=RegionOne) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:56.429745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39cfa2-ffc5-4095-ba81-16133979775e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:56.430647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9975-d5ed-4674-acba-26771d47a7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:56.431210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39cfa2-ffc5-4095-ba81-16133979775e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:56.431309 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9975-d5ed-4674-acba-26771d47a7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:56.431576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9975-d5ed-4674-acba-26771d47a7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:56.431847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9975-d5ed-4674-acba-26771d47a7a4 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:56.432555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db9975-d5ed-4674-acba-26771d47a7a4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:56.432859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db9975-d5ed-4674-acba-26771d47a7a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:56.434216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db9975-d5ed-4674-acba-26771d47a7a4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:56.437900 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 447/904] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 09:45:56 2022] GET /identity/v3/endpoints?service_id=8219d0daff7e4f5787b1ea57c17aa9d8&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:56.439130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39cfa2-ffc5-4095-ba81-16133979775e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:56.439281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 458/905] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 09:45:56 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:56.443785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9975-d5ed-4674-acba-26771d47a7a4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:56.564478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:56.579012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9975-d5ed-4674-acba-26771d47a7a4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:56.584015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39cfa2-ffc5-4095-ba81-16133979775e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:56.621588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22358696-a492-4dc7-993b-99884ffd74db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:56.622205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22358696-a492-4dc7-993b-99884ffd74db None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:56.622483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22358696-a492-4dc7-993b-99884ffd74db None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:56.623710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22358696-a492-4dc7-993b-99884ffd74db None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:56.623710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22358696-a492-4dc7-993b-99884ffd74db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:56.624982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22358696-a492-4dc7-993b-99884ffd74db None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:56.632746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 459/906] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 09:45:56 2022] GET /identity/v3/services => generated 1268 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:45:56.637400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52faabd5-792f-458d-bba3-859b867806b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:56.638238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faabd5-792f-458d-bba3-859b867806b4 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:56.638517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faabd5-792f-458d-bba3-859b867806b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:56.638838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faabd5-792f-458d-bba3-859b867806b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:56.648322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52faabd5-792f-458d-bba3-859b867806b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:56.650187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52faabd5-792f-458d-bba3-859b867806b4 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-52faabd5-792f-458d-bba3-859b867806b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:45:56.670629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:56.671602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:56.672764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:45:56.672764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:45:56.672764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:45:56.672764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:45:56.672764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:45:56.672764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:45:56.672764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:45:56.672764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:56.672764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 448/907] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:45:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:56.675731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22358696-a492-4dc7-993b-99884ffd74db None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:56.686298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:56.793643 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22358696-a492-4dc7-993b-99884ffd74db None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:56.834245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fcead-bf26-439a-b0af-c0b9aa25375c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:56.834898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fcead-bf26-439a-b0af-c0b9aa25375c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:56.835182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fcead-bf26-439a-b0af-c0b9aa25375c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:56.835459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fcead-bf26-439a-b0af-c0b9aa25375c None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:56.836275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fcead-bf26-439a-b0af-c0b9aa25375c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:56.836589 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fcead-bf26-439a-b0af-c0b9aa25375c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:56.838092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fcead-bf26-439a-b0af-c0b9aa25375c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:56.854214 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 460/908] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:45:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:56.860549 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52faabd5-792f-458d-bba3-859b867806b4 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:56.971335 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52faabd5-792f-458d-bba3-859b867806b4 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:57.009744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9076a-d77f-4342-ac52-840c9a49cb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:57.010418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9076a-d77f-4342-ac52-840c9a49cb50 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:57.010728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9076a-d77f-4342-ac52-840c9a49cb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:57.011012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9076a-d77f-4342-ac52-840c9a49cb50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:57.011853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9076a-d77f-4342-ac52-840c9a49cb50 None admin] RBAC: Adding query filter params (role.id=67c7f3afe8244828abe1191b2b97ae35, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=a9ac6a36e5224f38bdc5b214580ff18a) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:57.012167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9076a-d77f-4342-ac52-840c9a49cb50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:57.013617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9076a-d77f-4342-ac52-840c9a49cb50 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:57.026725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 449/909] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:45:56 2022] GET /identity/v3/role_assignments?user.id=a9ac6a36e5224f38bdc5b214580ff18a&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=67c7f3afe8244828abe1191b2b97ae35 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:57.699174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0f321-b9e7-4754-8ef4-6dfaae570a5a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:57.699709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0f321-b9e7-4754-8ef4-6dfaae570a5a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:57.699709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0f321-b9e7-4754-8ef4-6dfaae570a5a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:57.701289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 461/910] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:57.707986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf53bd-421f-490d-ade2-19c559f39304 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:57.708440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf53bd-421f-490d-ade2-19c559f39304 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:57.708536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf53bd-421f-490d-ade2-19c559f39304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:57.729924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-3edf53bd-421f-490d-ade2-19c559f39304 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:57.731152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3edf53bd-421f-490d-ade2-19c559f39304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:57.733337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:57.802616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 450/911] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:57.825807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0f321-b9e7-4754-8ef4-6dfaae570a5a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:57.900192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:57.937403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0f321-b9e7-4754-8ef4-6dfaae570a5a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:57.972041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab985df-caf2-4dad-91c0-5016f576bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:57.973087 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab985df-caf2-4dad-91c0-5016f576bafe None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:57.973893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab985df-caf2-4dad-91c0-5016f576bafe None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:57.974311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab985df-caf2-4dad-91c0-5016f576bafe None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:57.975138 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab985df-caf2-4dad-91c0-5016f576bafe None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:57.976658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab985df-caf2-4dad-91c0-5016f576bafe None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-6ab985df-caf2-4dad-91c0-5016f576bafe None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:57.989292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:57.990468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 09:45:57.991557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:57.994147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 462/912] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Jun 29 09:45:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:57.999951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf53bd-421f-490d-ade2-19c559f39304 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:58.103517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf53bd-421f-490d-ade2-19c559f39304 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:58.139244 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cb111-911c-4ae7-acd7-7091d8664542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:58.140258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cb111-911c-4ae7-acd7-7091d8664542 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:58.140734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cb111-911c-4ae7-acd7-7091d8664542 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:58.141442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cb111-911c-4ae7-acd7-7091d8664542 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:58.142350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cb111-911c-4ae7-acd7-7091d8664542 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:58.142861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cb111-911c-4ae7-acd7-7091d8664542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:58.144437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cb111-911c-4ae7-acd7-7091d8664542 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:58.150001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 451/913] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 09:45:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:58.159716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab985df-caf2-4dad-91c0-5016f576bafe None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:58.268531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab985df-caf2-4dad-91c0-5016f576bafe None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:58.298926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:58.303883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c47a6-f4d6-4c44-b03e-2ed15fec19d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:58.304536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c47a6-f4d6-4c44-b03e-2ed15fec19d3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:58.304821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c47a6-f4d6-4c44-b03e-2ed15fec19d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:58.305097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c47a6-f4d6-4c44-b03e-2ed15fec19d3 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:58.305817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c47a6-f4d6-4c44-b03e-2ed15fec19d3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:58.306127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c47a6-f4d6-4c44-b03e-2ed15fec19d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:58.307544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c47a6-f4d6-4c44-b03e-2ed15fec19d3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:58.313452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 463/914] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 09:45:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:58.319677 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cb111-911c-4ae7-acd7-7091d8664542 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:58.339377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:58.429386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cb111-911c-4ae7-acd7-7091d8664542 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:58.464132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858dfd9-dcba-4a2d-8832-e110f1f90d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:58.464805 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858dfd9-dcba-4a2d-8832-e110f1f90d54 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:58.465092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858dfd9-dcba-4a2d-8832-e110f1f90d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:58.465368 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858dfd9-dcba-4a2d-8832-e110f1f90d54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:58.466121 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858dfd9-dcba-4a2d-8832-e110f1f90d54 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:58.467526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858dfd9-dcba-4a2d-8832-e110f1f90d54 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:58.489804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 452/915] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 09:45:58 2022] POST /identity/v3/endpoints => generated 347 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:45:58.616437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545cc73-e2d7-4da8-af3a-f34e31df925f None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:58.616627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545cc73-e2d7-4da8-af3a-f34e31df925f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:58.616922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545cc73-e2d7-4da8-af3a-f34e31df925f None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:58.618593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 464/916] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:58.624320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff8897-f41a-4472-9e10-08b8cc50ef6a None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:58.624642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff8897-f41a-4472-9e10-08b8cc50ef6a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:58.624890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff8897-f41a-4472-9e10-08b8cc50ef6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:58.643549 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-baff8897-f41a-4472-9e10-08b8cc50ef6a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:58.644758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-baff8897-f41a-4472-9e10-08b8cc50ef6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:58.703742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:58.727500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 453/917] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:45:58.751947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545cc73-e2d7-4da8-af3a-f34e31df925f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:58.777904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad45da5-51c6-439a-b3d6-00cabe257aba None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:58.778231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad45da5-51c6-439a-b3d6-00cabe257aba None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:58.778451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad45da5-51c6-439a-b3d6-00cabe257aba None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:58.780128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 454/918] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:45:58.785801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005e998-7e77-4453-a642-2f5d7ca3c256 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:58.786352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005e998-7e77-4453-a642-2f5d7ca3c256 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:58.786846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005e998-7e77-4453-a642-2f5d7ca3c256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:58.804989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-1005e998-7e77-4453-a642-2f5d7ca3c256 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:58.806606 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-1005e998-7e77-4453-a642-2f5d7ca3c256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:58.862924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545cc73-e2d7-4da8-af3a-f34e31df925f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:58.886827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 455/919] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:45:58.901407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2bacc-65b1-4596-ae3f-5fbed446b222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:58.902094 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2bacc-65b1-4596-ae3f-5fbed446b222 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:58.902422 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2bacc-65b1-4596-ae3f-5fbed446b222 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:58.902728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2bacc-65b1-4596-ae3f-5fbed446b222 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:58.903547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2bacc-65b1-4596-ae3f-5fbed446b222 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:58.905014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2bacc-65b1-4596-ae3f-5fbed446b222 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:58.908429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005e998-7e77-4453-a642-2f5d7ca3c256 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:58.922781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 465/920] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:45:58 2022] POST /identity/v3/services => generated 241 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:45:59.032791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005e998-7e77-4453-a642-2f5d7ca3c256 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:59.065983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:59.079576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85db10-2a8d-4491-913e-76b122314d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:59.080395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85db10-2a8d-4491-913e-76b122314d38 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:59.080670 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85db10-2a8d-4491-913e-76b122314d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:59.080945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85db10-2a8d-4491-913e-76b122314d38 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:59.081602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85db10-2a8d-4491-913e-76b122314d38 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:59.083037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85db10-2a8d-4491-913e-76b122314d38 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-de85db10-2a8d-4491-913e-76b122314d38 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:59.095894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:59.098061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:59.099487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 456/921] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:59.104240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2bacc-65b1-4596-ae3f-5fbed446b222 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:59.168364 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:59.217767 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2bacc-65b1-4596-ae3f-5fbed446b222 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:59.230611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7812-be4b-4e6a-b682-607cb841fd6d None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:59.230901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7812-be4b-4e6a-b682-607cb841fd6d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:59.231177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7812-be4b-4e6a-b682-607cb841fd6d None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:59.232572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 457/922] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:45:59.239600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c90ba-391b-4ed7-a4a6-0c58b293527c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:59.239883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c90ba-391b-4ed7-a4a6-0c58b293527c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:59.240209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c90ba-391b-4ed7-a4a6-0c58b293527c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:59.258536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370b6ce-84ee-42bf-9e49-e9cf0201af02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:59.258536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370b6ce-84ee-42bf-9e49-e9cf0201af02 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:59.258536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370b6ce-84ee-42bf-9e49-e9cf0201af02 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:59.258536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370b6ce-84ee-42bf-9e49-e9cf0201af02 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:59.258987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370b6ce-84ee-42bf-9e49-e9cf0201af02 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:59.258987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370b6ce-84ee-42bf-9e49-e9cf0201af02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:59.258987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370b6ce-84ee-42bf-9e49-e9cf0201af02 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:59.263978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 466/923] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:59.264346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-620c90ba-391b-4ed7-a4a6-0c58b293527c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:45:59.270286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-620c90ba-391b-4ed7-a4a6-0c58b293527c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:45:59.271409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370b6ce-84ee-42bf-9e49-e9cf0201af02 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:59.353399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 458/924] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:45:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:45:59.376577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c90ba-391b-4ed7-a4a6-0c58b293527c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:59.391243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370b6ce-84ee-42bf-9e49-e9cf0201af02 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:59.410896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:59.425474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6fc96-75cf-4a40-ae35-b37b7e9334c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:59.426162 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6fc96-75cf-4a40-ae35-b37b7e9334c0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:59.426441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6fc96-75cf-4a40-ae35-b37b7e9334c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:59.426784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6fc96-75cf-4a40-ae35-b37b7e9334c0 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:59.427494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6fc96-75cf-4a40-ae35-b37b7e9334c0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:59.427800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6fc96-75cf-4a40-ae35-b37b7e9334c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:59.429237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6fc96-75cf-4a40-ae35-b37b7e9334c0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:59.434658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 467/925] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 09:45:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:59.440208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6fc96-75cf-4a40-ae35-b37b7e9334c0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:59.489310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c90ba-391b-4ed7-a4a6-0c58b293527c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:59.493411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:59.530050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e32a7-8f72-4054-8fd5-d0ab16527d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:59.530950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e32a7-8f72-4054-8fd5-d0ab16527d51 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:59.531387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e32a7-8f72-4054-8fd5-d0ab16527d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:59.531665 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e32a7-8f72-4054-8fd5-d0ab16527d51 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:59.540634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6fc96-75cf-4a40-ae35-b37b7e9334c0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:59.545551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e32a7-8f72-4054-8fd5-d0ab16527d51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-f38e32a7-8f72-4054-8fd5-d0ab16527d51 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:45:59.555601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:59.556584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:45:59.557435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 459/926] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 09:45:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:59.564024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e32a7-8f72-4054-8fd5-d0ab16527d51 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:59.582474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5317795-b65b-4ba7-9ad7-d9c2482f54d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:59.583303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317795-b65b-4ba7-9ad7-d9c2482f54d3 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:59.583715 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317795-b65b-4ba7-9ad7-d9c2482f54d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:59.584098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317795-b65b-4ba7-9ad7-d9c2482f54d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:59.584994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5317795-b65b-4ba7-9ad7-d9c2482f54d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:59.586516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5317795-b65b-4ba7-9ad7-d9c2482f54d3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:59.614904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 468/927] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 09:45:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:45:59.685546 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e32a7-8f72-4054-8fd5-d0ab16527d51 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:59.735008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b324142-18ab-4e23-9182-0c96d3a77164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:59.735695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b324142-18ab-4e23-9182-0c96d3a77164 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:59.735966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b324142-18ab-4e23-9182-0c96d3a77164 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:59.736240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b324142-18ab-4e23-9182-0c96d3a77164 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:59.737001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b324142-18ab-4e23-9182-0c96d3a77164 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:45:59.737311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b324142-18ab-4e23-9182-0c96d3a77164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:59.738756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b324142-18ab-4e23-9182-0c96d3a77164 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:59.753837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 460/928] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 09:45:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:45:59.755314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5317795-b65b-4ba7-9ad7-d9c2482f54d3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:59.799760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:45:59.863542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5317795-b65b-4ba7-9ad7-d9c2482f54d3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:45:59.898460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996a910-6dfa-4b98-8af5-b4d6c011c9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:45:59.899311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996a910-6dfa-4b98-8af5-b4d6c011c9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:45:59.899595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996a910-6dfa-4b98-8af5-b4d6c011c9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:45:59.899870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996a910-6dfa-4b98-8af5-b4d6c011c9e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:45:59.910195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996a910-6dfa-4b98-8af5-b4d6c011c9e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:45:59.911701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996a910-6dfa-4b98-8af5-b4d6c011c9e8 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-c996a910-6dfa-4b98-8af5-b4d6c011c9e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:45:59.929246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:45:59.930160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:45:59.931086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:45:59.931086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:45:59.931086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:45:59.931086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:45:59.931086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 469/929] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:45:59.938758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b324142-18ab-4e23-9182-0c96d3a77164 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:45:59.964315 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:00.048200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b324142-18ab-4e23-9182-0c96d3a77164 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:00.090051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd974a-5453-4d9c-994e-858f7b117e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:00.091154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd974a-5453-4d9c-994e-858f7b117e16 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:00.091638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd974a-5453-4d9c-994e-858f7b117e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:00.092131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd974a-5453-4d9c-994e-858f7b117e16 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:00.093081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd974a-5453-4d9c-994e-858f7b117e16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:00.093598 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd974a-5453-4d9c-994e-858f7b117e16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:00.096097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd974a-5453-4d9c-994e-858f7b117e16 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:00.110122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 461/930] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:00.115590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996a910-6dfa-4b98-8af5-b4d6c011c9e8 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:00.214247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:00.225732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996a910-6dfa-4b98-8af5-b4d6c011c9e8 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:00.260558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b677b-231b-4cde-af24-d8365ffc1b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:00.261373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b677b-231b-4cde-af24-d8365ffc1b52 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:00.261649 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b677b-231b-4cde-af24-d8365ffc1b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:00.261915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b677b-231b-4cde-af24-d8365ffc1b52 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:00.270512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b677b-231b-4cde-af24-d8365ffc1b52 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:00.271989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b677b-231b-4cde-af24-d8365ffc1b52 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-bd0b677b-231b-4cde-af24-d8365ffc1b52 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:46:00.287360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:00.288385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:00.289947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:00.289947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:00.289947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:46:00.289947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:46:00.289947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:46:00.289947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:46:00.289947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:46:00.289947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:46:00.289947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:46:00.289947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:00.289947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 470/931] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 09:46:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:00.294444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd974a-5453-4d9c-994e-858f7b117e16 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:00.380545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:00.401230 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd974a-5453-4d9c-994e-858f7b117e16 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:00.437726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c014716-6a01-4e3d-922e-5038dad1e868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:00.438615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c014716-6a01-4e3d-922e-5038dad1e868 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:00.439094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c014716-6a01-4e3d-922e-5038dad1e868 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:00.439373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c014716-6a01-4e3d-922e-5038dad1e868 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:00.440268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c014716-6a01-4e3d-922e-5038dad1e868 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:00.440577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c014716-6a01-4e3d-922e-5038dad1e868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:00.442060 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c014716-6a01-4e3d-922e-5038dad1e868 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:00.460088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 462/932] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 09:46:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:00.465731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b677b-231b-4cde-af24-d8365ffc1b52 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:00.570597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b677b-231b-4cde-af24-d8365ffc1b52 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:00.605263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd837b-5759-43f0-ad4c-3fde77689dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:00.606076 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd837b-5759-43f0-ad4c-3fde77689dca None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:00.606360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd837b-5759-43f0-ad4c-3fde77689dca None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:00.606664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd837b-5759-43f0-ad4c-3fde77689dca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:00.618448 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd837b-5759-43f0-ad4c-3fde77689dca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:00.619950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd837b-5759-43f0-ad4c-3fde77689dca None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-b9fd837b-5759-43f0-ad4c-3fde77689dca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:46:00.637103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:00.638089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:00.639277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:46:00.639277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:46:00.639277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:00.639277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:00.639277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 471/933] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:00.644746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c014716-6a01-4e3d-922e-5038dad1e868 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:00.752992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1728454-91dc-47c0-9ac9-e93457e4cc56 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:00.753181 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1728454-91dc-47c0-9ac9-e93457e4cc56 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:00.753586 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1728454-91dc-47c0-9ac9-e93457e4cc56 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:00.755329 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 472/934] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:00.760330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad6923-7d9c-481d-bfb5-447e8f5f0677 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:00.760698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad6923-7d9c-481d-bfb5-447e8f5f0677 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:00.760955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c014716-6a01-4e3d-922e-5038dad1e868 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:00.761347 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad6923-7d9c-481d-bfb5-447e8f5f0677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:00.778217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-56ad6923-7d9c-481d-bfb5-447e8f5f0677 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:00.779556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-56ad6923-7d9c-481d-bfb5-447e8f5f0677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:00.782946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:00.799107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f6585-4a50-4b26-ae54-5c345075382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:00.800121 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f6585-4a50-4b26-ae54-5c345075382c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:00.800577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f6585-4a50-4b26-ae54-5c345075382c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:00.801019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f6585-4a50-4b26-ae54-5c345075382c None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:00.801920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f6585-4a50-4b26-ae54-5c345075382c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:00.802393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f6585-4a50-4b26-ae54-5c345075382c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:00.803991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f6585-4a50-4b26-ae54-5c345075382c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:00.816761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 463/935] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:00.817161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:00.823164 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f6585-4a50-4b26-ae54-5c345075382c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:00.858832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 473/936] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:00.875401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad6923-7d9c-481d-bfb5-447e8f5f0677 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:00.932125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f6585-4a50-4b26-ae54-5c345075382c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:00.967692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5de155-36c4-46c6-a6fa-9ae1348cad2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:00.968852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5de155-36c4-46c6-a6fa-9ae1348cad2f None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:00.969410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5de155-36c4-46c6-a6fa-9ae1348cad2f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:00.969883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5de155-36c4-46c6-a6fa-9ae1348cad2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:00.977023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad6923-7d9c-481d-bfb5-447e8f5f0677 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:00.979133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5de155-36c4-46c6-a6fa-9ae1348cad2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:00.980803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5de155-36c4-46c6-a6fa-9ae1348cad2f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-8d5de155-36c4-46c6-a6fa-9ae1348cad2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:46:00.996490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:00.997433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:00.998328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:46:00.998328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:46:00.998328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:46:00.998328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:46:00.998328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:46:00.998328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:46:00.998328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:00.998328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:01.005455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 464/937] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:46:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:01.006505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5de155-36c4-46c6-a6fa-9ae1348cad2f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:01.006948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:01.015508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeb9f5-e923-4955-a531-6b4a47aac328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:01.016739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb9f5-e923-4955-a531-6b4a47aac328 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.017085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb9f5-e923-4955-a531-6b4a47aac328 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.017476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb9f5-e923-4955-a531-6b4a47aac328 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.018169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdeb9f5-e923-4955-a531-6b4a47aac328 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:01.020665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdeb9f5-e923-4955-a531-6b4a47aac328 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-ebdeb9f5-e923-4955-a531-6b4a47aac328 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:01.033296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:01.034320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 09:46:01.035195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:01.038754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 474/938] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:46:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:01.041009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeb9f5-e923-4955-a531-6b4a47aac328 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:01.082822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:01.129009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5de155-36c4-46c6-a6fa-9ae1348cad2f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:01.146771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeb9f5-e923-4955-a531-6b4a47aac328 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:01.165610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8246f-ef5a-4540-ab8f-9d6345646c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:01.166682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8246f-ef5a-4540-ab8f-9d6345646c0a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.167175 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8246f-ef5a-4540-ab8f-9d6345646c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.167642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8246f-ef5a-4540-ab8f-9d6345646c0a None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.168629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8246f-ef5a-4540-ab8f-9d6345646c0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:01.169145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8246f-ef5a-4540-ab8f-9d6345646c0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:01.170825 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8246f-ef5a-4540-ab8f-9d6345646c0a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:01.182529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02ccdc-cb74-4f22-a342-0c583116644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:01.183249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02ccdc-cb74-4f22-a342-0c583116644d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.183533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02ccdc-cb74-4f22-a342-0c583116644d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.183811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02ccdc-cb74-4f22-a342-0c583116644d None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.184566 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02ccdc-cb74-4f22-a342-0c583116644d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:01.184846 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02ccdc-cb74-4f22-a342-0c583116644d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:01.186232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02ccdc-cb74-4f22-a342-0c583116644d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:01.188646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 465/939] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:46:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:01.189833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454a5a8-a372-4245-bb09-26db940e8011 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.190121 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454a5a8-a372-4245-bb09-26db940e8011 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.190398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454a5a8-a372-4245-bb09-26db940e8011 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.191624 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 475/940] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:46:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:01.191727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 466/941] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:01.193061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02ccdc-cb74-4f22-a342-0c583116644d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:01.197038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d22ddf-1e0f-4e00-8419-5dff4e401a94 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.197320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d22ddf-1e0f-4e00-8419-5dff4e401a94 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.197660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d22ddf-1e0f-4e00-8419-5dff4e401a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.214413 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-26d22ddf-1e0f-4e00-8419-5dff4e401a94 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:01.215626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-26d22ddf-1e0f-4e00-8419-5dff4e401a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:01.259534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:01.292837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 467/942] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:01.295674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d22ddf-1e0f-4e00-8419-5dff4e401a94 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:01.310674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02ccdc-cb74-4f22-a342-0c583116644d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:01.358517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:01.358991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ace722-2355-40ae-b8c2-8cd4d10101a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:01.359969 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ace722-2355-40ae-b8c2-8cd4d10101a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.360253 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ace722-2355-40ae-b8c2-8cd4d10101a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.360526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ace722-2355-40ae-b8c2-8cd4d10101a0 None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/a9ac6a36e5224f38bdc5b214580ff18a/roles/67c7f3afe8244828abe1191b2b97ae35` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.364004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ace722-2355-40ae-b8c2-8cd4d10101a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=a9ac6a36e5224f38bdc5b214580ff18a, role_id=67c7f3afe8244828abe1191b2b97ae35)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:01.366322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ace722-2355-40ae-b8c2-8cd4d10101a0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:01.382877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 476/943] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:46:01 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/a9ac6a36e5224f38bdc5b214580ff18a/roles/67c7f3afe8244828abe1191b2b97ae35 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:46:01.383699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ace722-2355-40ae-b8c2-8cd4d10101a0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:01.423580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d22ddf-1e0f-4e00-8419-5dff4e401a94 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:01.479692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde9d80-6073-4bc9-8197-241baba7855d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:01.480388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde9d80-6073-4bc9-8197-241baba7855d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.480667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde9d80-6073-4bc9-8197-241baba7855d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.481001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde9d80-6073-4bc9-8197-241baba7855d None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.481738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde9d80-6073-4bc9-8197-241baba7855d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:01.482046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde9d80-6073-4bc9-8197-241baba7855d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:01.483731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde9d80-6073-4bc9-8197-241baba7855d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:01.490219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 468/944] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:46:01 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:01.523487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ace722-2355-40ae-b8c2-8cd4d10101a0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:01.550205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:01.569291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae43bb-a99c-435b-8fa1-829531df88f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:01.570451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae43bb-a99c-435b-8fa1-829531df88f5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.570917 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae43bb-a99c-435b-8fa1-829531df88f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.571405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae43bb-a99c-435b-8fa1-829531df88f5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.572268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae43bb-a99c-435b-8fa1-829531df88f5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:01.573834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae43bb-a99c-435b-8fa1-829531df88f5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-46ae43bb-a99c-435b-8fa1-829531df88f5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:01.587806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:01.589118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:01.590165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:01.592863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 477/945] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 09:46:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:01.599455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde9d80-6073-4bc9-8197-241baba7855d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:01.721799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde9d80-6073-4bc9-8197-241baba7855d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:01.738539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:01.761634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55265c6e-2be8-41ad-af62-2f764b541b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:01.763059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55265c6e-2be8-41ad-af62-2f764b541b91 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.763490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55265c6e-2be8-41ad-af62-2f764b541b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.763912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55265c6e-2be8-41ad-af62-2f764b541b91 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.764911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55265c6e-2be8-41ad-af62-2f764b541b91 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:01.765385 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55265c6e-2be8-41ad-af62-2f764b541b91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:01.766742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4fe58-6d6d-45a8-b5e9-c48ef3774c69 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.767147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4fe58-6d6d-45a8-b5e9-c48ef3774c69 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.767292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4fe58-6d6d-45a8-b5e9-c48ef3774c69 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.768012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55265c6e-2be8-41ad-af62-2f764b541b91 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:01.768917 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 478/946] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:01.773876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d888b-4b98-4e22-a811-e27193aa026d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.774089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d888b-4b98-4e22-a811-e27193aa026d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.774394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d888b-4b98-4e22-a811-e27193aa026d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.774821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 469/947] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 09:46:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:01.782184 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55265c6e-2be8-41ad-af62-2f764b541b91 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:01.792184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-195d888b-4b98-4e22-a811-e27193aa026d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:01.793359 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-195d888b-4b98-4e22-a811-e27193aa026d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:01.870665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 479/948] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:01.885688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55265c6e-2be8-41ad-af62-2f764b541b91 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:01.892340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d888b-4b98-4e22-a811-e27193aa026d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:01.921712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ec042-92a0-4de5-8a7e-53f20a56db0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:01.922425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ec042-92a0-4de5-8a7e-53f20a56db0f None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:01.922739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ec042-92a0-4de5-8a7e-53f20a56db0f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:01.923017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ec042-92a0-4de5-8a7e-53f20a56db0f None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:01.923730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ec042-92a0-4de5-8a7e-53f20a56db0f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:01.924040 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ec042-92a0-4de5-8a7e-53f20a56db0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:01.925418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ec042-92a0-4de5-8a7e-53f20a56db0f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:01.931002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 470/949] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 09:46:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:01.936578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ec042-92a0-4de5-8a7e-53f20a56db0f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:01.985467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:02.002547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d888b-4b98-4e22-a811-e27193aa026d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:02.025310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:02.054219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ec042-92a0-4de5-8a7e-53f20a56db0f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:02.056321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa129c-a28b-4a9a-b124-0f26c629ad94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:02.057281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa129c-a28b-4a9a-b124-0f26c629ad94 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:02.057763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa129c-a28b-4a9a-b124-0f26c629ad94 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:02.058236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa129c-a28b-4a9a-b124-0f26c629ad94 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:02.059197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa129c-a28b-4a9a-b124-0f26c629ad94 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:02.060863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa129c-a28b-4a9a-b124-0f26c629ad94 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:02.062552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: WARNING keystone.server.flask.common [None req-e0fa129c-a28b-4a9a-b124-0f26c629ad94 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 29 09:46:02.082473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.sql.core [None req-e0fa129c-a28b-4a9a-b124-0f26c629ad94 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 09:46:02.082473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 29 09:46:02.082473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [parameters: {'id': 'deefde8c5a7a46a1aadcd450637865f7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 29 09:46:02.082473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80177) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 09:46:02.095964 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedb726-c134-4a65-b552-737d7a3fed2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:02.097236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedb726-c134-4a65-b552-737d7a3fed2e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:02.097774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedb726-c134-4a65-b552-737d7a3fed2e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:02.098307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedb726-c134-4a65-b552-737d7a3fed2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:02.099633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedb726-c134-4a65-b552-737d7a3fed2e None admin] RBAC: Adding query filter params (interface=public, service_id=9287cd62bd2e49229d29f52fe7b2ca3c, region_id=RegionOne) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:02.100015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedb726-c134-4a65-b552-737d7a3fed2e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:02.101472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedb726-c134-4a65-b552-737d7a3fed2e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:02.107572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 471/950] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 09:46:01 2022] GET /identity/v3/endpoints?service_id=9287cd62bd2e49229d29f52fe7b2ca3c&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-e0fa129c-a28b-4a9a-b124-0f26c629ad94 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id deefde8c5a7a46a1aadcd450637865f7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id deefde8c5a7a46a1aadcd450637865f7. Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application conn.query(q) Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 09:46:02.107942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result.read() Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 29 09:46:02.108765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application next(self.gen) Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application yield resource Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application next(self.gen) Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise self.value Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application session.commit() Jun 29 09:46:02.109786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.session.flush() Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application compat.raise_( Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 09:46:02.110887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 09:46:02.111906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application conn.query(q) Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result.read() Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 09:46:02.112889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [parameters: {'id': 'deefde8c5a7a46a1aadcd450637865f7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:02.113976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:02.114876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:02.116019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:02.116019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 29 09:46:02.116019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 29 09:46:02.116019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:02.116019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:02.116019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 29 09:46:02.116019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 29 09:46:02.116019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id deefde8c5a7a46a1aadcd450637865f7. Jun 29 09:46:02.116019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:02.116019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 480/951] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Jun 29 09:46:01 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:02.117526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedb726-c134-4a65-b552-737d7a3fed2e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:02.118972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa129c-a28b-4a9a-b124-0f26c629ad94 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:02.246019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa129c-a28b-4a9a-b124-0f26c629ad94 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:02.251233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedb726-c134-4a65-b552-737d7a3fed2e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:02.262039 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:02.286534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748caeec-2ce0-4697-abd6-aa99119b9be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:02.287469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748caeec-2ce0-4697-abd6-aa99119b9be0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:02.287853 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748caeec-2ce0-4697-abd6-aa99119b9be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:02.288221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748caeec-2ce0-4697-abd6-aa99119b9be0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:02.296128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748caeec-2ce0-4697-abd6-aa99119b9be0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:02.298449 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748caeec-2ce0-4697-abd6-aa99119b9be0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:02.299577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d9d47-f7e6-4eeb-a21e-53ae927edeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:02.300682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9d47-f7e6-4eeb-a21e-53ae927edeae None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:02.301111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9d47-f7e6-4eeb-a21e-53ae927edeae None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:02.301935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9d47-f7e6-4eeb-a21e-53ae927edeae None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:02.302816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d9d47-f7e6-4eeb-a21e-53ae927edeae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:02.304321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d9d47-f7e6-4eeb-a21e-53ae927edeae None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:02.311441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 472/952] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 09:46:02 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-748caeec-2ce0-4697-abd6-aa99119b9be0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:46:02.313864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:02.315105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:02.316279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:46:02.316279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:46:02.316279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:46:02.316279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:46:02.316279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:46:02.316279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:46:02.316279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:02.316279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:02.318039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 481/953] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:46:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:02.327912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d9d47-f7e6-4eeb-a21e-53ae927edeae None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:02.443979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d9d47-f7e6-4eeb-a21e-53ae927edeae None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:02.482360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10faaa20-21bc-4204-b2c9-ebc63e2b653b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:02.483069 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faaa20-21bc-4204-b2c9-ebc63e2b653b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:02.483353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faaa20-21bc-4204-b2c9-ebc63e2b653b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:02.483629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faaa20-21bc-4204-b2c9-ebc63e2b653b None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:02.484391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10faaa20-21bc-4204-b2c9-ebc63e2b653b None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:02.484698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10faaa20-21bc-4204-b2c9-ebc63e2b653b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:02.486466 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10faaa20-21bc-4204-b2c9-ebc63e2b653b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:02.505428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 473/954] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:46:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:03.570127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96637968-8b7c-48c1-8774-60ae7b355b2a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:03.570713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96637968-8b7c-48c1-8774-60ae7b355b2a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:03.570713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96637968-8b7c-48c1-8774-60ae7b355b2a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:03.575071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 482/955] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:03.580409 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018f9c7-6e3e-4163-b2a0-f21d205aa2e3 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:03.580809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018f9c7-6e3e-4163-b2a0-f21d205aa2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:03.581065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018f9c7-6e3e-4163-b2a0-f21d205aa2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:03.598733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2018f9c7-6e3e-4163-b2a0-f21d205aa2e3 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:03.599907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2018f9c7-6e3e-4163-b2a0-f21d205aa2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:03.625072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:03.673973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b30bc-0230-4d90-b1fd-b2fe3ff7e8be None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:03.674748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b30bc-0230-4d90-b1fd-b2fe3ff7e8be None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:03.675258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b30bc-0230-4d90-b1fd-b2fe3ff7e8be None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:03.677421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 483/956] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:03.681309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 474/957] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:03.684115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb2c7-1aa2-41ca-980e-540e1aa36c6c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:03.684638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb2c7-1aa2-41ca-980e-540e1aa36c6c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:03.685216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb2c7-1aa2-41ca-980e-540e1aa36c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:03.700836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018f9c7-6e3e-4163-b2a0-f21d205aa2e3 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:03.706223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-656cb2c7-1aa2-41ca-980e-540e1aa36c6c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:03.708263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-656cb2c7-1aa2-41ca-980e-540e1aa36c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:03.748683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:03.807834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 484/958] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:03.827516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018f9c7-6e3e-4163-b2a0-f21d205aa2e3 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:03.829123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cb2c7-1aa2-41ca-980e-540e1aa36c6c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:03.866062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2e4fa-9ed3-4572-bf43-1baf88fa93a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:03.867284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2e4fa-9ed3-4572-bf43-1baf88fa93a2 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:03.867810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2e4fa-9ed3-4572-bf43-1baf88fa93a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:03.868272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2e4fa-9ed3-4572-bf43-1baf88fa93a2 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:03.869254 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2e4fa-9ed3-4572-bf43-1baf88fa93a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:03.870928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2e4fa-9ed3-4572-bf43-1baf88fa93a2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:03.894947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 475/959] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:46:03 2022] POST /identity/v3/services => generated 242 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:46:03.950693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cb2c7-1aa2-41ca-980e-540e1aa36c6c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:03.997628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae665a3a-c0c3-4f13-bdf8-c949abf2f9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:03.998934 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae665a3a-c0c3-4f13-bdf8-c949abf2f9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:03.999419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae665a3a-c0c3-4f13-bdf8-c949abf2f9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:03.999889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae665a3a-c0c3-4f13-bdf8-c949abf2f9e4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:04.012314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae665a3a-c0c3-4f13-bdf8-c949abf2f9e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-ae665a3a-c0c3-4f13-bdf8-c949abf2f9e4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:46:04.021279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:04.022350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:46:04.023243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:04.026271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 485/960] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 09:46:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:04.032915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2e4fa-9ed3-4572-bf43-1baf88fa93a2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:04.064314 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:04.159147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2e4fa-9ed3-4572-bf43-1baf88fa93a2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:04.194499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582aa7c7-9d70-4c13-aa90-5905ef32af23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:04.195598 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582aa7c7-9d70-4c13-aa90-5905ef32af23 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:04.196102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582aa7c7-9d70-4c13-aa90-5905ef32af23 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:04.196601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582aa7c7-9d70-4c13-aa90-5905ef32af23 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:04.197552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582aa7c7-9d70-4c13-aa90-5905ef32af23 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:04.198070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582aa7c7-9d70-4c13-aa90-5905ef32af23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:04.199712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582aa7c7-9d70-4c13-aa90-5905ef32af23 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:04.209614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 476/961] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 09:46:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:04.215672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae665a3a-c0c3-4f13-bdf8-c949abf2f9e4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:04.237000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:04.327195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae665a3a-c0c3-4f13-bdf8-c949abf2f9e4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:04.362786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6133085-7821-40da-9059-b688147732b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:04.363597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133085-7821-40da-9059-b688147732b3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:04.363871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133085-7821-40da-9059-b688147732b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:04.364139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133085-7821-40da-9059-b688147732b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:04.375356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6133085-7821-40da-9059-b688147732b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:04.377127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6133085-7821-40da-9059-b688147732b3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-e6133085-7821-40da-9059-b688147732b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:46:04.394684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:04.395671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:04.396571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:46:04.396571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:46:04.396571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:04.396571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:04.396571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 486/962] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:04.403083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582aa7c7-9d70-4c13-aa90-5905ef32af23 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:04.432248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a1834-b9ff-4083-9c56-8f34122001a1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:04.432525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a1834-b9ff-4083-9c56-8f34122001a1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:04.432856 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a1834-b9ff-4083-9c56-8f34122001a1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:04.434209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 487/963] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:46:04.439750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115748a7-e7ac-4c80-855c-c8b6219e47df None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:04.441130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115748a7-e7ac-4c80-855c-c8b6219e47df None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:04.441414 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115748a7-e7ac-4c80-855c-c8b6219e47df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:04.460033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-115748a7-e7ac-4c80-855c-c8b6219e47df None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:04.461275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-115748a7-e7ac-4c80-855c-c8b6219e47df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:04.533092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:04.553483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582aa7c7-9d70-4c13-aa90-5905ef32af23 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:04.555226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:04.571240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 488/964] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:04 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:04.593306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82138d12-5b09-4983-a4ea-2075805c432e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:04.594986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82138d12-5b09-4983-a4ea-2075805c432e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:04.595477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82138d12-5b09-4983-a4ea-2075805c432e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:04.595969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82138d12-5b09-4983-a4ea-2075805c432e None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:04.596937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82138d12-5b09-4983-a4ea-2075805c432e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:04.597455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82138d12-5b09-4983-a4ea-2075805c432e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:04.599645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82138d12-5b09-4983-a4ea-2075805c432e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:04.602467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115748a7-e7ac-4c80-855c-c8b6219e47df None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:04.614159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 477/965] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:04.620001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82138d12-5b09-4983-a4ea-2075805c432e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:04.722817 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82138d12-5b09-4983-a4ea-2075805c432e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:04.760296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0be7fd-3a44-4e6c-8690-ea4cd0114ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:04.761127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0be7fd-3a44-4e6c-8690-ea4cd0114ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:04.761405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0be7fd-3a44-4e6c-8690-ea4cd0114ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:04.761673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0be7fd-3a44-4e6c-8690-ea4cd0114ab1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:04.761769 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115748a7-e7ac-4c80-855c-c8b6219e47df None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:04.771091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0be7fd-3a44-4e6c-8690-ea4cd0114ab1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:04.773495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0be7fd-3a44-4e6c-8690-ea4cd0114ab1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-4c0be7fd-3a44-4e6c-8690-ea4cd0114ab1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:46:04.794455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:04.795643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:04.796789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:04.796789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:04.796789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:46:04.796789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:46:04.796789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:46:04.796789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:46:04.796789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:46:04.796789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:46:04.796789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 09:46:04.796789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:04.801730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 478/966] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 09:46:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:04.802946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad0c02-82ec-4e16-8b02-14538f4ed141 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:04.803235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad0c02-82ec-4e16-8b02-14538f4ed141 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:04.803508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad0c02-82ec-4e16-8b02-14538f4ed141 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:04.806435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 479/967] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:04.806435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad0c02-82ec-4e16-8b02-14538f4ed141 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:04.808990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:04.824652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced90546-22ab-4eb3-be9a-78f9a2ceea21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:04.825485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced90546-22ab-4eb3-be9a-78f9a2ceea21 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:04.825764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced90546-22ab-4eb3-be9a-78f9a2ceea21 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:04.826041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced90546-22ab-4eb3-be9a-78f9a2ceea21 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:04.826758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced90546-22ab-4eb3-be9a-78f9a2ceea21 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:04.828158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced90546-22ab-4eb3-be9a-78f9a2ceea21 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-ced90546-22ab-4eb3-be9a-78f9a2ceea21 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:04.841200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:04.842706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:04.843652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 489/968] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 09:46:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:04.844792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb3cb0-f715-4607-aeb6-169c8beadfb5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:04.844792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb3cb0-f715-4607-aeb6-169c8beadfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:04.844792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb3cb0-f715-4607-aeb6-169c8beadfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:04.863012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-35eb3cb0-f715-4607-aeb6-169c8beadfb5 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:04.864233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-35eb3cb0-f715-4607-aeb6-169c8beadfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:04.929339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad0c02-82ec-4e16-8b02-14538f4ed141 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:04.941962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:04.958620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 490/969] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:04 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:04.959122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb3cb0-f715-4607-aeb6-169c8beadfb5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:04.971114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61743a-2cab-4f75-890d-389fe6bc09e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:04.972034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61743a-2cab-4f75-890d-389fe6bc09e6 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:04.972578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61743a-2cab-4f75-890d-389fe6bc09e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:04.973067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61743a-2cab-4f75-890d-389fe6bc09e6 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:04.974442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61743a-2cab-4f75-890d-389fe6bc09e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:04.974922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61743a-2cab-4f75-890d-389fe6bc09e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:04.976531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61743a-2cab-4f75-890d-389fe6bc09e6 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:04.995124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 480/970] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 09:46:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:04.996226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61743a-2cab-4f75-890d-389fe6bc09e6 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:05.060735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:05.114605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61743a-2cab-4f75-890d-389fe6bc09e6 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:05.116622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb3cb0-f715-4607-aeb6-169c8beadfb5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:05.158582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72673c7a-df46-45b6-b37a-ccdc0ffc1484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:05.159295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72673c7a-df46-45b6-b37a-ccdc0ffc1484 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:05.159756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72673c7a-df46-45b6-b37a-ccdc0ffc1484 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:05.159863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72673c7a-df46-45b6-b37a-ccdc0ffc1484 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:05.160587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72673c7a-df46-45b6-b37a-ccdc0ffc1484 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:05.160868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72673c7a-df46-45b6-b37a-ccdc0ffc1484 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:05.162287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72673c7a-df46-45b6-b37a-ccdc0ffc1484 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:05.166237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a20209-250d-4da0-9d98-c31c85258d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:05.167126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a20209-250d-4da0-9d98-c31c85258d87 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:05.167421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a20209-250d-4da0-9d98-c31c85258d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:05.167645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a20209-250d-4da0-9d98-c31c85258d87 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:05.168376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a20209-250d-4da0-9d98-c31c85258d87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:05.169878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a20209-250d-4da0-9d98-c31c85258d87 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:05.171321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: WARNING keystone.server.flask.common [None req-69a20209-250d-4da0-9d98-c31c85258d87 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 29 09:46:05.177625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 491/971] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 09:46:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:05.178867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72673c7a-df46-45b6-b37a-ccdc0ffc1484 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:05.185132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.sql.core [None req-69a20209-250d-4da0-9d98-c31c85258d87 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 09:46:05.185132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 29 09:46:05.185132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [parameters: {'user_id': '45f5342be6764b6ebeef70489d8e7dc7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 29 09:46:05.185132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80176) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-69a20209-250d-4da0-9d98-c31c85258d87 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application conn.query(q) Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application result.read() Jun 29 09:46:05.192692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 09:46:05.193629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application next(self.gen) Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application yield resource Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application next(self.gen) Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise self.value Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application session.commit() Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 09:46:05.195616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self.session.flush() Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application compat.raise_( Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 09:46:05.196536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 09:46:05.197493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application conn.query(q) Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application result.read() Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 09:46:05.198354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [parameters: {'user_id': '45f5342be6764b6ebeef70489d8e7dc7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:05.199288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 29 09:46:05.200108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 29 09:46:05.201007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 29 09:46:05.201007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 29 09:46:05.201007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 09:46:05.201007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 09:46:05.201007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 29 09:46:05.201007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:05.201007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 481/972] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jun 29 09:46:04 2022] POST /identity/v3/users => generated 160 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:05.201007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a20209-250d-4da0-9d98-c31c85258d87 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:05.275101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:05.309111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a20209-250d-4da0-9d98-c31c85258d87 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:05.335437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:05.336659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72673c7a-df46-45b6-b37a-ccdc0ffc1484 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:05.353654 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d416b-dd78-4575-9196-d60414781047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:05.355070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d416b-dd78-4575-9196-d60414781047 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:05.355671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d416b-dd78-4575-9196-d60414781047 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:05.356330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d416b-dd78-4575-9196-d60414781047 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:05.357499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d416b-dd78-4575-9196-d60414781047 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:05.357934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d416b-dd78-4575-9196-d60414781047 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:05.360637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d416b-dd78-4575-9196-d60414781047 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:05.373204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 482/973] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 09:46:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:05.374465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d416b-dd78-4575-9196-d60414781047 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:05.380356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae447eac-19d8-4f78-8ea8-61fa04ba95a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:05.381027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae447eac-19d8-4f78-8ea8-61fa04ba95a4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:05.381305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae447eac-19d8-4f78-8ea8-61fa04ba95a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:05.381582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae447eac-19d8-4f78-8ea8-61fa04ba95a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:05.467664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae447eac-19d8-4f78-8ea8-61fa04ba95a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:05.469217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae447eac-19d8-4f78-8ea8-61fa04ba95a4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:05.529857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d416b-dd78-4575-9196-d60414781047 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:05.568354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:05.583112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e58bbe-b321-46c4-8079-71b457b4d9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:05.584346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e58bbe-b321-46c4-8079-71b457b4d9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:05.584753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e58bbe-b321-46c4-8079-71b457b4d9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:05.585373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e58bbe-b321-46c4-8079-71b457b4d9e8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:05.595159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e58bbe-b321-46c4-8079-71b457b4d9e8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:05.596625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e58bbe-b321-46c4-8079-71b457b4d9e8 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-74e58bbe-b321-46c4-8079-71b457b4d9e8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:46:05.620521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:05.622618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:05.624741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:05.624741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:05.624741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:46:05.624741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:46:05.624741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:46:05.624741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:46:05.624741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:46:05.624741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:46:05.624741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 09:46:05.624741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:05.630115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 483/974] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:46:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:05.631481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e58bbe-b321-46c4-8079-71b457b4d9e8 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:05.637554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 492/975] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:46:05 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:46:05.638985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae447eac-19d8-4f78-8ea8-61fa04ba95a4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:05.712843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:05.777467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae447eac-19d8-4f78-8ea8-61fa04ba95a4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:05.778159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e58bbe-b321-46c4-8079-71b457b4d9e8 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:05.817436 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9337d-34da-4805-8c12-2df4003e9267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:05.818020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9337d-34da-4805-8c12-2df4003e9267 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:05.818301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9337d-34da-4805-8c12-2df4003e9267 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:05.818591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9337d-34da-4805-8c12-2df4003e9267 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:05.819346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9337d-34da-4805-8c12-2df4003e9267 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:05.820725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9337d-34da-4805-8c12-2df4003e9267 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:05.831058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a874f-3847-4fe4-86bd-d0fc48565b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:05.831750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a874f-3847-4fe4-86bd-d0fc48565b97 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:05.832031 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a874f-3847-4fe4-86bd-d0fc48565b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:05.832306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a874f-3847-4fe4-86bd-d0fc48565b97 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:05.833124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a874f-3847-4fe4-86bd-d0fc48565b97 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:05.833430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a874f-3847-4fe4-86bd-d0fc48565b97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:05.837867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a874f-3847-4fe4-86bd-d0fc48565b97 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:05.854988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 484/976] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 09:46:05 2022] POST /identity/v3/endpoints => generated 349 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:46:05.854988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9337d-34da-4805-8c12-2df4003e9267 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:05.889205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 493/977] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 09:46:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:05.987781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:05.994599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9337d-34da-4805-8c12-2df4003e9267 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:06.037457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3a318-28a9-4d1b-b522-976ec56b44ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:06.038297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3a318-28a9-4d1b-b522-976ec56b44ca None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:06.038596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3a318-28a9-4d1b-b522-976ec56b44ca None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:06.038988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3a318-28a9-4d1b-b522-976ec56b44ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:06.055714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3a318-28a9-4d1b-b522-976ec56b44ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:06.056695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3a318-28a9-4d1b-b522-976ec56b44ca None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-beb3a318-28a9-4d1b-b522-976ec56b44ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:46:06.078903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:06.079852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:06.081101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:46:06.081101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:46:06.081101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:06.081101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:06.081101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 485/978] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:06.087630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a874f-3847-4fe4-86bd-d0fc48565b97 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:06.103683 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d265-256b-46a2-9672-832e5a2f6e71 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:06.104010 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d265-256b-46a2-9672-832e5a2f6e71 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:06.104232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d265-256b-46a2-9672-832e5a2f6e71 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:06.105790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 486/979] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:46:06.106357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:06.111447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534459-493d-4edd-b978-298b04a6ebaf None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:06.111754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534459-493d-4edd-b978-298b04a6ebaf None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:06.112006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534459-493d-4edd-b978-298b04a6ebaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:06.136327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-68534459-493d-4edd-b978-298b04a6ebaf None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:06.138296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-68534459-493d-4edd-b978-298b04a6ebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:06.218816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a874f-3847-4fe4-86bd-d0fc48565b97 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:06.230889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 487/980] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:06.255144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68534459-493d-4edd-b978-298b04a6ebaf None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:06.266647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71796239-55d4-4a84-9639-4f560a1549ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:06.267864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71796239-55d4-4a84-9639-4f560a1549ce None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:06.268350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71796239-55d4-4a84-9639-4f560a1549ce None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:06.268805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71796239-55d4-4a84-9639-4f560a1549ce None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:06.269843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71796239-55d4-4a84-9639-4f560a1549ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:06.270366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71796239-55d4-4a84-9639-4f560a1549ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:06.271999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71796239-55d4-4a84-9639-4f560a1549ce None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:06.286488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 494/981] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:06.292036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71796239-55d4-4a84-9639-4f560a1549ce None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:06.340962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:06.361279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68534459-493d-4edd-b978-298b04a6ebaf None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:06.370795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:06.399674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71796239-55d4-4a84-9639-4f560a1549ce None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:06.400555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d8fd7-83ba-470f-802f-e9029480e2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:06.401377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d8fd7-83ba-470f-802f-e9029480e2ed None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:06.401657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d8fd7-83ba-470f-802f-e9029480e2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:06.401936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d8fd7-83ba-470f-802f-e9029480e2ed None admin] PATH_INFO: `/v3/services/compute` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:06.402627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d8fd7-83ba-470f-802f-e9029480e2ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:06.404017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d8fd7-83ba-470f-802f-e9029480e2ed None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-2f3d8fd7-83ba-470f-802f-e9029480e2ed None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:06.416653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:06.417754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:06.418633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 488/982] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:46:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:06.424348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d8fd7-83ba-470f-802f-e9029480e2ed None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:06.442398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5c13b-93f7-4974-96e0-7b01212652bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:06.443070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c13b-93f7-4974-96e0-7b01212652bb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:06.443348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c13b-93f7-4974-96e0-7b01212652bb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:06.443617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c13b-93f7-4974-96e0-7b01212652bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:06.451014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5c13b-93f7-4974-96e0-7b01212652bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:06.452504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5c13b-93f7-4974-96e0-7b01212652bb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-88b5c13b-93f7-4974-96e0-7b01212652bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:46:06.468212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:06.469393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:06.470291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:46:06.470291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:46:06.470291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:46:06.470291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:46:06.470291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:46:06.470291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:46:06.470291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:06.470291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:06.470291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 495/983] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:06.476830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5c13b-93f7-4974-96e0-7b01212652bb None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:06.548210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d8fd7-83ba-470f-802f-e9029480e2ed None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:06.590140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497db8f-b047-4cce-8dc5-917f5841f3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:06.590897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497db8f-b047-4cce-8dc5-917f5841f3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:06.591233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497db8f-b047-4cce-8dc5-917f5841f3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:06.591453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497db8f-b047-4cce-8dc5-917f5841f3e9 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:06.591689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5c13b-93f7-4974-96e0-7b01212652bb None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:06.592169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497db8f-b047-4cce-8dc5-917f5841f3e9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:06.592476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497db8f-b047-4cce-8dc5-917f5841f3e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:06.593855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497db8f-b047-4cce-8dc5-917f5841f3e9 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:06.602147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 489/984] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:46:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:06.608591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497db8f-b047-4cce-8dc5-917f5841f3e9 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:06.622961 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:06.639262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd92f1-07af-40bf-9b9a-8368a25c8190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:06.639996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd92f1-07af-40bf-9b9a-8368a25c8190 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:06.640218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd92f1-07af-40bf-9b9a-8368a25c8190 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:06.640493 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd92f1-07af-40bf-9b9a-8368a25c8190 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:06.641427 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd92f1-07af-40bf-9b9a-8368a25c8190 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:06.641736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd92f1-07af-40bf-9b9a-8368a25c8190 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:06.643230 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd92f1-07af-40bf-9b9a-8368a25c8190 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:06.645218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:06.661792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 496/985] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:46:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:06.663441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd92f1-07af-40bf-9b9a-8368a25c8190 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:06.736602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497db8f-b047-4cce-8dc5-917f5841f3e9 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:06.770011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd92f1-07af-40bf-9b9a-8368a25c8190 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:06.773042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f1c0f-97f9-4b06-8d02-86633942c569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:06.773881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f1c0f-97f9-4b06-8d02-86633942c569 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:06.774301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f1c0f-97f9-4b06-8d02-86633942c569 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:06.774758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f1c0f-97f9-4b06-8d02-86633942c569 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:06.775604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f1c0f-97f9-4b06-8d02-86633942c569 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:06.776065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f1c0f-97f9-4b06-8d02-86633942c569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:06.777702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f1c0f-97f9-4b06-8d02-86633942c569 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:06.788447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 490/986] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:46:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:06.789497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f1c0f-97f9-4b06-8d02-86633942c569 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:06.810328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df4cc9-a2d3-4cdd-a513-cb3975149aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:06.811021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4cc9-a2d3-4cdd-a513-cb3975149aff None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:06.811299 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4cc9-a2d3-4cdd-a513-cb3975149aff None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:06.811567 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4cc9-a2d3-4cdd-a513-cb3975149aff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:06.812399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df4cc9-a2d3-4cdd-a513-cb3975149aff None admin] RBAC: Adding query filter params (role.id=67c7f3afe8244828abe1191b2b97ae35, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=a9ac6a36e5224f38bdc5b214580ff18a) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:06.812706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df4cc9-a2d3-4cdd-a513-cb3975149aff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:06.814531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df4cc9-a2d3-4cdd-a513-cb3975149aff None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:06.820711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 497/987] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:46:06 2022] GET /identity/v3/role_assignments?user.id=a9ac6a36e5224f38bdc5b214580ff18a&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=67c7f3afe8244828abe1191b2b97ae35 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:06.871468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:06.906291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f1c0f-97f9-4b06-8d02-86633942c569 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:06.944410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ce947-7820-41f5-8b63-3a44fc98ba77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:06.945105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ce947-7820-41f5-8b63-3a44fc98ba77 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:06.945386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ce947-7820-41f5-8b63-3a44fc98ba77 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:06.945660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ce947-7820-41f5-8b63-3a44fc98ba77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:06.946470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ce947-7820-41f5-8b63-3a44fc98ba77 None admin] RBAC: Adding query filter params (interface=public, service_id=a517ed4cbdf94fd6ad305c61a1667ddc, region_id=RegionOne) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:06.946812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ce947-7820-41f5-8b63-3a44fc98ba77 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:06.948229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ce947-7820-41f5-8b63-3a44fc98ba77 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:06.954553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 491/988] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 09:46:06 2022] GET /identity/v3/endpoints?service_id=a517ed4cbdf94fd6ad305c61a1667ddc&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:06.960117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df4cc9-a2d3-4cdd-a513-cb3975149aff None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:07.024608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:07.075761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df4cc9-a2d3-4cdd-a513-cb3975149aff None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:07.113014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beee322-8761-410a-8587-7fc2b6847608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:07.113697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beee322-8761-410a-8587-7fc2b6847608 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:07.113975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beee322-8761-410a-8587-7fc2b6847608 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:07.114251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beee322-8761-410a-8587-7fc2b6847608 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:07.114967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beee322-8761-410a-8587-7fc2b6847608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:07.116353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beee322-8761-410a-8587-7fc2b6847608 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:07.123352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 498/989] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 09:46:06 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:46:07.996892 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d2c74-399c-43b5-8152-fec95e2425e9 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:07.997459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d2c74-399c-43b5-8152-fec95e2425e9 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:07.997459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d2c74-399c-43b5-8152-fec95e2425e9 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:07.999036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 492/990] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:08.004452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162cb10-1228-4a98-af50-288c8c6d5c65 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:08.004906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162cb10-1228-4a98-af50-288c8c6d5c65 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:08.004984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162cb10-1228-4a98-af50-288c8c6d5c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:08.022879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5162cb10-1228-4a98-af50-288c8c6d5c65 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:08.024225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5162cb10-1228-4a98-af50-288c8c6d5c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:08.103424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 499/991] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:08.126539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d2c74-399c-43b5-8152-fec95e2425e9 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:08.243220 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d2c74-399c-43b5-8152-fec95e2425e9 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:08.281072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d368e-9add-4a93-9b1a-c84a12f69e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:08.281867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d368e-9add-4a93-9b1a-c84a12f69e1e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:08.282150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d368e-9add-4a93-9b1a-c84a12f69e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:08.282425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d368e-9add-4a93-9b1a-c84a12f69e1e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:08.283116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d368e-9add-4a93-9b1a-c84a12f69e1e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:08.284493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d368e-9add-4a93-9b1a-c84a12f69e1e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:08.287466 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-cb3d368e-9add-4a93-9b1a-c84a12f69e1e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:08.298906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:08.300215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:08.301268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:08.304046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 493/992] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 09:46:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:08.310157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162cb10-1228-4a98-af50-288c8c6d5c65 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:08.364468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:08.414283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162cb10-1228-4a98-af50-288c8c6d5c65 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:08.450546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397298a-214c-4a35-a750-811d411b40f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:08.451256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397298a-214c-4a35-a750-811d411b40f9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:08.451541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397298a-214c-4a35-a750-811d411b40f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:08.451820 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397298a-214c-4a35-a750-811d411b40f9 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:08.452537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397298a-214c-4a35-a750-811d411b40f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:08.452847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397298a-214c-4a35-a750-811d411b40f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:08.454279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397298a-214c-4a35-a750-811d411b40f9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:08.459734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 500/993] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 09:46:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:08.465548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d368e-9add-4a93-9b1a-c84a12f69e1e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:08.571988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d368e-9add-4a93-9b1a-c84a12f69e1e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:08.608859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67db7a0-da3a-4e40-9f4c-59a30f92c707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:08.609544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db7a0-da3a-4e40-9f4c-59a30f92c707 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:08.609864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db7a0-da3a-4e40-9f4c-59a30f92c707 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:08.610141 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db7a0-da3a-4e40-9f4c-59a30f92c707 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:08.610905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67db7a0-da3a-4e40-9f4c-59a30f92c707 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:08.611225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67db7a0-da3a-4e40-9f4c-59a30f92c707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:08.612635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67db7a0-da3a-4e40-9f4c-59a30f92c707 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:08.618666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 494/994] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 09:46:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:08.624723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397298a-214c-4a35-a750-811d411b40f9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:08.738866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397298a-214c-4a35-a750-811d411b40f9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:08.755675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:08.776076 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d598b-fbcc-41a1-85ec-0fe6bde59d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:08.777029 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d598b-fbcc-41a1-85ec-0fe6bde59d9e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:08.777459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d598b-fbcc-41a1-85ec-0fe6bde59d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:08.777877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d598b-fbcc-41a1-85ec-0fe6bde59d9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:08.778869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d598b-fbcc-41a1-85ec-0fe6bde59d9e None admin] RBAC: Adding query filter params (interface=admin, service_id=9287cd62bd2e49229d29f52fe7b2ca3c, region_id=RegionOne) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:08.779330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d598b-fbcc-41a1-85ec-0fe6bde59d9e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:08.780889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d598b-fbcc-41a1-85ec-0fe6bde59d9e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:08.787503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 501/995] 10.222.0.52 () {56 vars in 1249 bytes} [Wed Jun 29 09:46:08 2022] GET /identity/v3/endpoints?service_id=9287cd62bd2e49229d29f52fe7b2ca3c&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:08.793155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67db7a0-da3a-4e40-9f4c-59a30f92c707 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:08.861621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:08.903723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67db7a0-da3a-4e40-9f4c-59a30f92c707 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:08.935904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59dd75-c2d6-4089-b1ae-5dc66a23dcf3 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:08.936181 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59dd75-c2d6-4089-b1ae-5dc66a23dcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:08.936475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59dd75-c2d6-4089-b1ae-5dc66a23dcf3 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:08.938029 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 502/996] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:08.939053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb38b9b-ac43-4562-91be-ffa84c7f9014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:08.939730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb38b9b-ac43-4562-91be-ffa84c7f9014 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:08.940012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb38b9b-ac43-4562-91be-ffa84c7f9014 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:08.940287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb38b9b-ac43-4562-91be-ffa84c7f9014 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:08.940988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb38b9b-ac43-4562-91be-ffa84c7f9014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:08.942852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb38b9b-ac43-4562-91be-ffa84c7f9014 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:08.943285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dafd7-e561-4058-82f4-8baeb48f9533 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:08.943573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dafd7-e561-4058-82f4-8baeb48f9533 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:08.943841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dafd7-e561-4058-82f4-8baeb48f9533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:08.958840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 495/997] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 09:46:08 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:46:08.961363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c19dafd7-e561-4058-82f4-8baeb48f9533 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:08.962620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c19dafd7-e561-4058-82f4-8baeb48f9533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:09.044949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 503/998] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:09.067901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb38b9b-ac43-4562-91be-ffa84c7f9014 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:09.175121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebbee6-5777-47c1-8f5d-7cb7c5877f5d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:09.175301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebbee6-5777-47c1-8f5d-7cb7c5877f5d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:09.175594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebbee6-5777-47c1-8f5d-7cb7c5877f5d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:09.176971 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 504/999] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:46:09.181952 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e31d80-5617-44f5-a76b-0c2402cfba9b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:09.182233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e31d80-5617-44f5-a76b-0c2402cfba9b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:09.182505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e31d80-5617-44f5-a76b-0c2402cfba9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:09.184110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb38b9b-ac43-4562-91be-ffa84c7f9014 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:09.199198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e0e31d80-5617-44f5-a76b-0c2402cfba9b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:09.200453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e0e31d80-5617-44f5-a76b-0c2402cfba9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:09.215440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:09.224275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebdf65-d04c-4690-8c66-6da91747173d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:09.225102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebdf65-d04c-4690-8c66-6da91747173d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:09.225376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebdf65-d04c-4690-8c66-6da91747173d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:09.225651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebdf65-d04c-4690-8c66-6da91747173d None admin] PATH_INFO: `/v3/services/image` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:09.226318 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebdf65-d04c-4690-8c66-6da91747173d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:09.227845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebdf65-d04c-4690-8c66-6da91747173d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-93ebdf65-d04c-4690-8c66-6da91747173d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:09.240360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:09.241509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:09.247749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 496/1000] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Jun 29 09:46:09 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:09.248719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebdf65-d04c-4690-8c66-6da91747173d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:09.278766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 505/1001] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:09.302158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e31d80-5617-44f5-a76b-0c2402cfba9b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:09.350659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:09.364141 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebdf65-d04c-4690-8c66-6da91747173d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:09.401962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6036eb-128c-48c8-8f1c-f53a061b1d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:09.402673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6036eb-128c-48c8-8f1c-f53a061b1d76 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:09.402956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6036eb-128c-48c8-8f1c-f53a061b1d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:09.403226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6036eb-128c-48c8-8f1c-f53a061b1d76 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:09.403934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6036eb-128c-48c8-8f1c-f53a061b1d76 None admin] RBAC: Adding query filter params (name=image) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:09.404235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6036eb-128c-48c8-8f1c-f53a061b1d76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:09.405609 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6036eb-128c-48c8-8f1c-f53a061b1d76 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:09.416472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 497/1002] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:09.418377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6036eb-128c-48c8-8f1c-f53a061b1d76 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:09.420879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e31d80-5617-44f5-a76b-0c2402cfba9b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:09.459997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035705-6888-48b3-9281-c76b56750610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:09.460811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035705-6888-48b3-9281-c76b56750610 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:09.461093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035705-6888-48b3-9281-c76b56750610 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:09.461365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035705-6888-48b3-9281-c76b56750610 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:09.462024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f035705-6888-48b3-9281-c76b56750610 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:09.463428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f035705-6888-48b3-9281-c76b56750610 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-6f035705-6888-48b3-9281-c76b56750610 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:09.475918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:09.476998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:09.477838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 506/1003] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:46:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:09.483177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035705-6888-48b3-9281-c76b56750610 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:09.483332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:09.533697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6036eb-128c-48c8-8f1c-f53a061b1d76 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:09.569884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e61ce0-31ad-49ef-a7b5-aab06d3534c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:09.570666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61ce0-31ad-49ef-a7b5-aab06d3534c4 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:09.570942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61ce0-31ad-49ef-a7b5-aab06d3534c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:09.571148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61ce0-31ad-49ef-a7b5-aab06d3534c4 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:09.571853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e61ce0-31ad-49ef-a7b5-aab06d3534c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:09.572150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e61ce0-31ad-49ef-a7b5-aab06d3534c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:09.573528 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e61ce0-31ad-49ef-a7b5-aab06d3534c4 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:09.578306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 498/1004] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:09 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:09.602736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035705-6888-48b3-9281-c76b56750610 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:09.606533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:09.662401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e53684-b487-437a-aec6-156b247dff2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:09.663627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e53684-b487-437a-aec6-156b247dff2d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:09.664128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e53684-b487-437a-aec6-156b247dff2d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:09.664622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e53684-b487-437a-aec6-156b247dff2d None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:09.666010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e53684-b487-437a-aec6-156b247dff2d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:09.666587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e53684-b487-437a-aec6-156b247dff2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:09.669458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e53684-b487-437a-aec6-156b247dff2d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:09.679531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 507/1005] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:46:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:09.688940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e61ce0-31ad-49ef-a7b5-aab06d3534c4 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:09.823480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e61ce0-31ad-49ef-a7b5-aab06d3534c4 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:09.844455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:09.861444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab6811-7bf9-4a40-b256-e364cff500df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:09.862121 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab6811-7bf9-4a40-b256-e364cff500df None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:09.862402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab6811-7bf9-4a40-b256-e364cff500df None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:09.862704 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab6811-7bf9-4a40-b256-e364cff500df None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:09.863412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab6811-7bf9-4a40-b256-e364cff500df None admin] RBAC: Adding query filter params (type=compute) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:09.863721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab6811-7bf9-4a40-b256-e364cff500df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:09.865096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab6811-7bf9-4a40-b256-e364cff500df None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:09.870386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 499/1006] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 09:46:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:09.876218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e53684-b487-437a-aec6-156b247dff2d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:09.980451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e53684-b487-437a-aec6-156b247dff2d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:10.015474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e91c5-4043-45ae-81cf-eac129dd1b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:10.016159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e91c5-4043-45ae-81cf-eac129dd1b88 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:10.016443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e91c5-4043-45ae-81cf-eac129dd1b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:10.016710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e91c5-4043-45ae-81cf-eac129dd1b88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:10.017464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e91c5-4043-45ae-81cf-eac129dd1b88 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:10.018887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e91c5-4043-45ae-81cf-eac129dd1b88 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:10.047875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 508/1007] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 09:46:09 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:46:11.043269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ddec7-8ad7-485a-9bbb-c4de629772c4 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:11.043801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ddec7-8ad7-485a-9bbb-c4de629772c4 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:11.043801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ddec7-8ad7-485a-9bbb-c4de629772c4 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:11.045234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 500/1008] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:11.051081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00580d52-f2f3-4031-be98-67e837cd9344 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:11.051571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00580d52-f2f3-4031-be98-67e837cd9344 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:11.051858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00580d52-f2f3-4031-be98-67e837cd9344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:11.069187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-00580d52-f2f3-4031-be98-67e837cd9344 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:11.070706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-00580d52-f2f3-4031-be98-67e837cd9344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:11.088950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:11.154599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 509/1009] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:11 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:11.174541 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ddec7-8ad7-485a-9bbb-c4de629772c4 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:11.287456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ddec7-8ad7-485a-9bbb-c4de629772c4 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:11.324597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa1fa7-6f0d-4187-83f5-26b0d55e86df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:11.325320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1fa7-6f0d-4187-83f5-26b0d55e86df None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:11.325592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1fa7-6f0d-4187-83f5-26b0d55e86df None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:11.325858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1fa7-6f0d-4187-83f5-26b0d55e86df None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:11.326515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa1fa7-6f0d-4187-83f5-26b0d55e86df None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:11.327902 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa1fa7-6f0d-4187-83f5-26b0d55e86df None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-e0aa1fa7-6f0d-4187-83f5-26b0d55e86df None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:11.340496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:11.341497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:11.347980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 501/1010] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 09:46:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:11.350882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00580d52-f2f3-4031-be98-67e837cd9344 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:11.452007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00580d52-f2f3-4031-be98-67e837cd9344 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:11.486688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e8a98-be16-4f7e-8707-3c1926ec748d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:11.487361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e8a98-be16-4f7e-8707-3c1926ec748d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:11.487638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e8a98-be16-4f7e-8707-3c1926ec748d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:11.487906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e8a98-be16-4f7e-8707-3c1926ec748d None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:11.488612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e8a98-be16-4f7e-8707-3c1926ec748d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:11.488913 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e8a98-be16-4f7e-8707-3c1926ec748d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:11.490292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e8a98-be16-4f7e-8707-3c1926ec748d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:11.495419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 510/1011] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 09:46:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:11.501306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa1fa7-6f0d-4187-83f5-26b0d55e86df None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:11.554719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:11.609552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa1fa7-6f0d-4187-83f5-26b0d55e86df None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:11.627923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40352d-ea02-4cfb-ac4f-6c27cbc67d7d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:11.628204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40352d-ea02-4cfb-ac4f-6c27cbc67d7d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:11.628474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40352d-ea02-4cfb-ac4f-6c27cbc67d7d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:11.630108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 511/1012] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:46:11.635031 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a09411-2f60-46bb-af33-790802ba57b0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:11.635317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a09411-2f60-46bb-af33-790802ba57b0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:11.635589 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a09411-2f60-46bb-af33-790802ba57b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:11.648429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6a47e-7505-4e99-9726-f47819a27d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:11.649110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a47e-7505-4e99-9726-f47819a27d0e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:11.649395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a47e-7505-4e99-9726-f47819a27d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:11.649663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a47e-7505-4e99-9726-f47819a27d0e None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:11.650371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6a47e-7505-4e99-9726-f47819a27d0e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:11.650708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6a47e-7505-4e99-9726-f47819a27d0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:11.652206 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6a47e-7505-4e99-9726-f47819a27d0e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:11.653084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-58a09411-2f60-46bb-af33-790802ba57b0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:11.654278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-58a09411-2f60-46bb-af33-790802ba57b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:11.660151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 502/1013] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 09:46:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:11.665673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6a47e-7505-4e99-9726-f47819a27d0e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:11.668557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:11.738388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 512/1014] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:11 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:11.761584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a09411-2f60-46bb-af33-790802ba57b0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:11.791744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6a47e-7505-4e99-9726-f47819a27d0e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:11.807481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:11.832348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105f27d-cec4-40cd-ab4e-2811609455a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:11.833015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105f27d-cec4-40cd-ab4e-2811609455a2 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:11.833289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105f27d-cec4-40cd-ab4e-2811609455a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:11.833557 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105f27d-cec4-40cd-ab4e-2811609455a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:11.834297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105f27d-cec4-40cd-ab4e-2811609455a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:11.835716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105f27d-cec4-40cd-ab4e-2811609455a2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:11.862685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 503/1015] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 09:46:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:46:11.881407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a09411-2f60-46bb-af33-790802ba57b0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:11.926692 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5761e6-2b1b-4f87-9253-452c5b66ba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:11.927383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5761e6-2b1b-4f87-9253-452c5b66ba5d None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:11.927667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5761e6-2b1b-4f87-9253-452c5b66ba5d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:11.927940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5761e6-2b1b-4f87-9253-452c5b66ba5d None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:11.928676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5761e6-2b1b-4f87-9253-452c5b66ba5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:11.930131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5761e6-2b1b-4f87-9253-452c5b66ba5d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:11.943028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:11.950666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 513/1016] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:46:11 2022] POST /identity/v3/services => generated 242 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:46:12.193241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b816ab-787a-475a-99bf-73b3e7258787 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:12.193764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b816ab-787a-475a-99bf-73b3e7258787 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:12.193843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b816ab-787a-475a-99bf-73b3e7258787 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:12.195419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 504/1017] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:12.200542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3b0ae-c7c8-4b32-807f-76837b2d1736 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:12.201011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3b0ae-c7c8-4b32-807f-76837b2d1736 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:12.201091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3b0ae-c7c8-4b32-807f-76837b2d1736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:12.218135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5cb3b0ae-c7c8-4b32-807f-76837b2d1736 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:12.219463 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5cb3b0ae-c7c8-4b32-807f-76837b2d1736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:12.297807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 514/1018] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:12.321776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b816ab-787a-475a-99bf-73b3e7258787 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:12.434620 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b816ab-787a-475a-99bf-73b3e7258787 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:12.474237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8380e-e102-47d7-b0fe-183e31719da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:12.475000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8380e-e102-47d7-b0fe-183e31719da3 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:12.475276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8380e-e102-47d7-b0fe-183e31719da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:12.475543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8380e-e102-47d7-b0fe-183e31719da3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:12.487431 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8380e-e102-47d7-b0fe-183e31719da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-fef8380e-e102-47d7-b0fe-183e31719da3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:46:12.495526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:12.496483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:12.497352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 505/1019] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 09:46:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:12.502996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3b0ae-c7c8-4b32-807f-76837b2d1736 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:12.606980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3b0ae-c7c8-4b32-807f-76837b2d1736 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:12.634143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:12.643791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5331f-8d54-4db3-92a5-99acf2ef21d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:12.644482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5331f-8d54-4db3-92a5-99acf2ef21d5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:12.644764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5331f-8d54-4db3-92a5-99acf2ef21d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:12.645040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5331f-8d54-4db3-92a5-99acf2ef21d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:12.645792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5331f-8d54-4db3-92a5-99acf2ef21d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:12.650654 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5331f-8d54-4db3-92a5-99acf2ef21d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:12.650654 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5331f-8d54-4db3-92a5-99acf2ef21d5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:12.663766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 515/1020] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 09:46:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:12.665016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8380e-e102-47d7-b0fe-183e31719da3 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:12.699241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:12.776367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8380e-e102-47d7-b0fe-183e31719da3 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:12.810429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3d022-00e9-4acc-9431-c127f488b659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:12.811264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3d022-00e9-4acc-9431-c127f488b659 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:12.811548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3d022-00e9-4acc-9431-c127f488b659 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:12.811824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3d022-00e9-4acc-9431-c127f488b659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:12.822361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3d022-00e9-4acc-9431-c127f488b659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:12.823863 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3d022-00e9-4acc-9431-c127f488b659 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-d9f3d022-00e9-4acc-9431-c127f488b659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:46:12.841785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:12.842950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:12.848926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:46:12.848926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:46:12.848926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:12.848926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:12.848926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 506/1021] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:12.850285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5331f-8d54-4db3-92a5-99acf2ef21d5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:12.956171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5331f-8d54-4db3-92a5-99acf2ef21d5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:12.992061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfb237-1b26-4864-9deb-6975cb39e704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:12.992736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfb237-1b26-4864-9deb-6975cb39e704 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:12.993017 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfb237-1b26-4864-9deb-6975cb39e704 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:12.993287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfb237-1b26-4864-9deb-6975cb39e704 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:12.994010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfb237-1b26-4864-9deb-6975cb39e704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:12.994315 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfb237-1b26-4864-9deb-6975cb39e704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:12.995752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfb237-1b26-4864-9deb-6975cb39e704 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:13.007940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 516/1022] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:13.013558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3d022-00e9-4acc-9431-c127f488b659 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:13.115915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:13.133777 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3d022-00e9-4acc-9431-c127f488b659 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:13.169599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f783072-6c67-469f-9b57-d17aa9c33fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:13.170720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f783072-6c67-469f-9b57-d17aa9c33fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:13.171162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f783072-6c67-469f-9b57-d17aa9c33fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:13.171577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f783072-6c67-469f-9b57-d17aa9c33fa4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:13.179615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f783072-6c67-469f-9b57-d17aa9c33fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:13.181231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f783072-6c67-469f-9b57-d17aa9c33fa4 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-3f783072-6c67-469f-9b57-d17aa9c33fa4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:46:13.196195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:13.197603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:13.203718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:13.203718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:13.203718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:46:13.203718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:46:13.203718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:46:13.203718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:46:13.203718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:46:13.203718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:46:13.203718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:46:13.203718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:13.203718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 507/1023] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 09:46:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:13.204560 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfb237-1b26-4864-9deb-6975cb39e704 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:13.257568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:13.310646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfb237-1b26-4864-9deb-6975cb39e704 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:13.345933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b6bd6-d860-41bf-98bf-f21e403c4ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:13.346616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b6bd6-d860-41bf-98bf-f21e403c4ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:13.346906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b6bd6-d860-41bf-98bf-f21e403c4ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:13.347178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b6bd6-d860-41bf-98bf-f21e403c4ae5 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:13.348008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b6bd6-d860-41bf-98bf-f21e403c4ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:13.348676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b6bd6-d860-41bf-98bf-f21e403c4ae5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:13.350114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b6bd6-d860-41bf-98bf-f21e403c4ae5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:13.367701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 517/1024] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 09:46:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:13.373366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f783072-6c67-469f-9b57-d17aa9c33fa4 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:13.476840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f783072-6c67-469f-9b57-d17aa9c33fa4 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:13.512597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a224d-5c71-41f3-9b6e-4104a80b6bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:13.513250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a224d-5c71-41f3-9b6e-4104a80b6bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:13.513533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a224d-5c71-41f3-9b6e-4104a80b6bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:13.513804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a224d-5c71-41f3-9b6e-4104a80b6bd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:13.529319 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:13.588502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a224d-5c71-41f3-9b6e-4104a80b6bd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:13.590196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a224d-5c71-41f3-9b6e-4104a80b6bd9 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:13.696725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 508/1025] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:46:13 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:46:13.702186 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b6bd6-d860-41bf-98bf-f21e403c4ae5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:13.813065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b6bd6-d860-41bf-98bf-f21e403c4ae5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:13.823171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:13.854197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71023778-5701-44f3-9464-f630916788a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:13.854946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71023778-5701-44f3-9464-f630916788a2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:13.855225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71023778-5701-44f3-9464-f630916788a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:13.855495 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71023778-5701-44f3-9464-f630916788a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:13.866209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71023778-5701-44f3-9464-f630916788a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:13.867697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71023778-5701-44f3-9464-f630916788a2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-71023778-5701-44f3-9464-f630916788a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:46:13.886165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:13.887197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:13.888091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:46:13.888091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:46:13.888091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:13.888091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:13.888862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 518/1026] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:13.893410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a224d-5c71-41f3-9b6e-4104a80b6bd9 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:13.941951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:13.957355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132756d-9dca-4adb-9a38-4267f726461a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:13.957720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132756d-9dca-4adb-9a38-4267f726461a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:13.957903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132756d-9dca-4adb-9a38-4267f726461a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:13.960076 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 519/1027] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:13.964837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9591b-1c21-4946-831a-5b36323c9049 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:13.965133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9591b-1c21-4946-831a-5b36323c9049 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:13.965407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9591b-1c21-4946-831a-5b36323c9049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:13.983323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-85f9591b-1c21-4946-831a-5b36323c9049 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:13.984536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-85f9591b-1c21-4946-831a-5b36323c9049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:14.007240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a224d-5c71-41f3-9b6e-4104a80b6bd9 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:14.043047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09f1d1-9fbf-4418-a030-e840ad51a3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:14.043732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09f1d1-9fbf-4418-a030-e840ad51a3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.044017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09f1d1-9fbf-4418-a030-e840ad51a3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.044295 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09f1d1-9fbf-4418-a030-e840ad51a3c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.045031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09f1d1-9fbf-4418-a030-e840ad51a3c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:14.045341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09f1d1-9fbf-4418-a030-e840ad51a3c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:14.046750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09f1d1-9fbf-4418-a030-e840ad51a3c5 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:14.057444 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 520/1028] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:14.058834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d1190-2506-4c74-a2ae-3b7ee2fff294 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.059117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d1190-2506-4c74-a2ae-3b7ee2fff294 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.059390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d1190-2506-4c74-a2ae-3b7ee2fff294 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.060308 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 509/1029] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:14.064574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 521/1030] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:14.066511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e745f77-c327-4ede-9772-a5fca560dc31 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.066831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e745f77-c327-4ede-9772-a5fca560dc31 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.067106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d1190-2506-4c74-a2ae-3b7ee2fff294 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:14.067227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e745f77-c327-4ede-9772-a5fca560dc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.083710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6e745f77-c327-4ede-9772-a5fca560dc31 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:14.084907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6e745f77-c327-4ede-9772-a5fca560dc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:14.152767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 510/1031] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:14.153493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e745f77-c327-4ede-9772-a5fca560dc31 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:14.159152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:14.174489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d1190-2506-4c74-a2ae-3b7ee2fff294 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:14.198075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:14.209200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba7b2d-08ad-4260-b6ed-62fba69dc70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:14.210016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba7b2d-08ad-4260-b6ed-62fba69dc70a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.210299 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba7b2d-08ad-4260-b6ed-62fba69dc70a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.210585 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba7b2d-08ad-4260-b6ed-62fba69dc70a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.217515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba7b2d-08ad-4260-b6ed-62fba69dc70a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:14.219082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba7b2d-08ad-4260-b6ed-62fba69dc70a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-a5ba7b2d-08ad-4260-b6ed-62fba69dc70a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:46:14.232982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:14.233915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:14.234983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:46:14.234983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:46:14.234983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:46:14.234983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:46:14.234983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:46:14.234983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:46:14.234983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:14.234983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:14.234983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 522/1032] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:46:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:14.235485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba7b2d-08ad-4260-b6ed-62fba69dc70a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:14.269182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e745f77-c327-4ede-9772-a5fca560dc31 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:14.303516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a5447-b2ff-4e7f-93c1-237247dc30d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:14.304320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a5447-b2ff-4e7f-93c1-237247dc30d9 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.304605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a5447-b2ff-4e7f-93c1-237247dc30d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.304874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a5447-b2ff-4e7f-93c1-237247dc30d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.306043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a5447-b2ff-4e7f-93c1-237247dc30d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:14.307530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a5447-b2ff-4e7f-93c1-237247dc30d9 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:14.309169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 511/1033] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:14.309927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a5447-b2ff-4e7f-93c1-237247dc30d9 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:14.340051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba7b2d-08ad-4260-b6ed-62fba69dc70a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:14.374066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f62a3-da1d-4f29-8432-86e8763755e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:14.374890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f62a3-da1d-4f29-8432-86e8763755e9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.375209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f62a3-da1d-4f29-8432-86e8763755e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.375486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f62a3-da1d-4f29-8432-86e8763755e9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.376138 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f62a3-da1d-4f29-8432-86e8763755e9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:14.377542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f62a3-da1d-4f29-8432-86e8763755e9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-089f62a3-da1d-4f29-8432-86e8763755e9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:14.389134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:14.390338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:14.391477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 523/1034] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Jun 29 09:46:14 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:14.392309 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f62a3-da1d-4f29-8432-86e8763755e9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:14.406750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:14.416128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a5447-b2ff-4e7f-93c1-237247dc30d9 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:14.417705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:14.453525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c1c74-216f-44fc-a746-e05bb392a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:14.454217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c1c74-216f-44fc-a746-e05bb392a532 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.454496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c1c74-216f-44fc-a746-e05bb392a532 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.454790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c1c74-216f-44fc-a746-e05bb392a532 None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.455576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c1c74-216f-44fc-a746-e05bb392a532 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:14.455885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c1c74-216f-44fc-a746-e05bb392a532 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:14.457328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c1c74-216f-44fc-a746-e05bb392a532 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:14.469432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 512/1035] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:46:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:14.470253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c1c74-216f-44fc-a746-e05bb392a532 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:14.494757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f62a3-da1d-4f29-8432-86e8763755e9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:14.528109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34061d69-fe4e-42e3-b206-c8418b6e9500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:14.528780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34061d69-fe4e-42e3-b206-c8418b6e9500 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.529056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34061d69-fe4e-42e3-b206-c8418b6e9500 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.529328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34061d69-fe4e-42e3-b206-c8418b6e9500 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.530029 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34061d69-fe4e-42e3-b206-c8418b6e9500 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:14.531555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34061d69-fe4e-42e3-b206-c8418b6e9500 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:14.558429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 524/1036] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:46:14 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:46:14.559322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34061d69-fe4e-42e3-b206-c8418b6e9500 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:14.584151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c1c74-216f-44fc-a746-e05bb392a532 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:14.625805 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e34f48-1232-4241-b2ea-bee20c5f4b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:14.626802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e34f48-1232-4241-b2ea-bee20c5f4b56 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.627082 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e34f48-1232-4241-b2ea-bee20c5f4b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.627542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e34f48-1232-4241-b2ea-bee20c5f4b56 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.628711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e34f48-1232-4241-b2ea-bee20c5f4b56 None admin] RBAC: Adding query filter params (name=image) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:14.629030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e34f48-1232-4241-b2ea-bee20c5f4b56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:14.630935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e34f48-1232-4241-b2ea-bee20c5f4b56 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:14.634723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:14.639236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 513/1037] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:14.645565 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e34f48-1232-4241-b2ea-bee20c5f4b56 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:14.677335 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:14.682381 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34061d69-fe4e-42e3-b206-c8418b6e9500 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:14.724256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cee86b-abb0-4dc9-bf11-cdfa07acaa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:14.724929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cee86b-abb0-4dc9-bf11-cdfa07acaa1d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.725216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cee86b-abb0-4dc9-bf11-cdfa07acaa1d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.725552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cee86b-abb0-4dc9-bf11-cdfa07acaa1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.726413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cee86b-abb0-4dc9-bf11-cdfa07acaa1d None admin] RBAC: Adding query filter params (role.id=67c7f3afe8244828abe1191b2b97ae35, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=8a5eea7f539048c48ed10e72dd933df9) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:14.726788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cee86b-abb0-4dc9-bf11-cdfa07acaa1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:14.728356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cee86b-abb0-4dc9-bf11-cdfa07acaa1d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:14.735532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 525/1038] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:46:14 2022] GET /identity/v3/role_assignments?user.id=8a5eea7f539048c48ed10e72dd933df9&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=67c7f3afe8244828abe1191b2b97ae35 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:14.772518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e34f48-1232-4241-b2ea-bee20c5f4b56 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:14.812342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73425d-8aec-4325-814e-783a5b037ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:14.813129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73425d-8aec-4325-814e-783a5b037ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.813411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73425d-8aec-4325-814e-783a5b037ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.813685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73425d-8aec-4325-814e-783a5b037ad0 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.814417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73425d-8aec-4325-814e-783a5b037ad0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:14.814759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73425d-8aec-4325-814e-783a5b037ad0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:14.816191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73425d-8aec-4325-814e-783a5b037ad0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:14.822349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 514/1039] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:14.828503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cee86b-abb0-4dc9-bf11-cdfa07acaa1d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:14.942957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cee86b-abb0-4dc9-bf11-cdfa07acaa1d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:14.969767 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:14.979458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26312036-473d-4de8-9c9c-a43aa38adda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:14.980139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26312036-473d-4de8-9c9c-a43aa38adda6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:14.980422 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26312036-473d-4de8-9c9c-a43aa38adda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:14.980693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26312036-473d-4de8-9c9c-a43aa38adda6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:14.981493 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26312036-473d-4de8-9c9c-a43aa38adda6 None admin] RBAC: Adding query filter params (interface=public, service_id=98070e95b02147539737103577501927, region_id=RegionOne) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:14.981864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26312036-473d-4de8-9c9c-a43aa38adda6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:14.983293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26312036-473d-4de8-9c9c-a43aa38adda6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:14.993701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 526/1040] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 09:46:14 2022] GET /identity/v3/endpoints?service_id=98070e95b02147539737103577501927&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:14.995391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73425d-8aec-4325-814e-783a5b037ad0 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:15.102973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:15.113840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73425d-8aec-4325-814e-783a5b037ad0 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:15.150436 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaef04-3415-4eb3-95dd-48cb597d7b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:15.151143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaef04-3415-4eb3-95dd-48cb597d7b8b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:15.151421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaef04-3415-4eb3-95dd-48cb597d7b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:15.151694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaef04-3415-4eb3-95dd-48cb597d7b8b None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:15.152386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaef04-3415-4eb3-95dd-48cb597d7b8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:15.153758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaef04-3415-4eb3-95dd-48cb597d7b8b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:15.167052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 515/1041] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 09:46:14 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:46:16.647958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8d60-aa9e-47e5-84f2-18eed0ef1e9a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:16.648606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8d60-aa9e-47e5-84f2-18eed0ef1e9a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:16.648606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8d60-aa9e-47e5-84f2-18eed0ef1e9a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:16.650043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 527/1042] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:16.655317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40c258-a619-4b42-87f8-e95307009a01 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:16.655790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40c258-a619-4b42-87f8-e95307009a01 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:16.655868 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40c258-a619-4b42-87f8-e95307009a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:16.673457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-8c40c258-a619-4b42-87f8-e95307009a01 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:16.674673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8c40c258-a619-4b42-87f8-e95307009a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:16.747906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 516/1043] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:16.772172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e8d60-aa9e-47e5-84f2-18eed0ef1e9a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:16.791431 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5eab3-b9f1-458c-b517-6793f81ff6a5 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:16.791775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5eab3-b9f1-458c-b517-6793f81ff6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:16.791990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5eab3-b9f1-458c-b517-6793f81ff6a5 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:16.793535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 517/1044] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:16.798539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed671d62-f59b-4b3e-9993-b20dedbbb041 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:16.798889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed671d62-f59b-4b3e-9993-b20dedbbb041 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:16.799124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed671d62-f59b-4b3e-9993-b20dedbbb041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:16.816060 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-ed671d62-f59b-4b3e-9993-b20dedbbb041 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:16.817235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ed671d62-f59b-4b3e-9993-b20dedbbb041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:16.882793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e8d60-aa9e-47e5-84f2-18eed0ef1e9a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:16.893928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 518/1045] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:16.914255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed671d62-f59b-4b3e-9993-b20dedbbb041 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:16.917396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139661b-bf76-4502-95aa-c3ae92114413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:16.918208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139661b-bf76-4502-95aa-c3ae92114413 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:16.918484 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139661b-bf76-4502-95aa-c3ae92114413 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:16.918792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139661b-bf76-4502-95aa-c3ae92114413 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:16.919993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139661b-bf76-4502-95aa-c3ae92114413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:16.921428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139661b-bf76-4502-95aa-c3ae92114413 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:16.923297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 528/1046] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:16.929058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139661b-bf76-4502-95aa-c3ae92114413 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:16.929535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:16.994634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:17.017813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed671d62-f59b-4b3e-9993-b20dedbbb041 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:17.036828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139661b-bf76-4502-95aa-c3ae92114413 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:17.059128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bad85-b2ef-40fe-aa9d-0486c8c811ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:17.060491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bad85-b2ef-40fe-aa9d-0486c8c811ff None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.060781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bad85-b2ef-40fe-aa9d-0486c8c811ff None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.061053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bad85-b2ef-40fe-aa9d-0486c8c811ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.072606 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bad85-b2ef-40fe-aa9d-0486c8c811ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:46:17.080001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a0ccc-bfd3-47b4-948f-9dc6561427ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-8b9bad85-b2ef-40fe-aa9d-0486c8c811ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:46:17.080931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:17.081858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:46:17.082739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:17.083372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a0ccc-bfd3-47b4-948f-9dc6561427ff None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.083442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 519/1047] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 09:46:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:17.083509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a0ccc-bfd3-47b4-948f-9dc6561427ff None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.083509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a0ccc-bfd3-47b4-948f-9dc6561427ff None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.084142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a0ccc-bfd3-47b4-948f-9dc6561427ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:17.085197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a0ccc-bfd3-47b4-948f-9dc6561427ff None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:17.088236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bad85-b2ef-40fe-aa9d-0486c8c811ff None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:17.109117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 529/1048] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:46:16 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:46:17.176255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:17.202529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec169f-6fe1-4f0d-b2d8-953aa3a41c06 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.202967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec169f-6fe1-4f0d-b2d8-953aa3a41c06 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.203281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec169f-6fe1-4f0d-b2d8-953aa3a41c06 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.204955 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 530/1049] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:46:17.207456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bad85-b2ef-40fe-aa9d-0486c8c811ff None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:17.210808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596d230-540b-4398-9ebc-12e590d51771 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.211093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596d230-540b-4398-9ebc-12e590d51771 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.211368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596d230-540b-4398-9ebc-12e590d51771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.229682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d596d230-540b-4398-9ebc-12e590d51771 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:17.230964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d596d230-540b-4398-9ebc-12e590d51771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:17.248610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143aeed-4dae-4a06-8383-1c5722ce0b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:17.249310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143aeed-4dae-4a06-8383-1c5722ce0b66 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.249599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143aeed-4dae-4a06-8383-1c5722ce0b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.249867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143aeed-4dae-4a06-8383-1c5722ce0b66 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.250780 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143aeed-4dae-4a06-8383-1c5722ce0b66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:17.251091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143aeed-4dae-4a06-8383-1c5722ce0b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:17.252611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143aeed-4dae-4a06-8383-1c5722ce0b66 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:17.267614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 520/1050] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 09:46:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:17.268736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143aeed-4dae-4a06-8383-1c5722ce0b66 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:17.312072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 531/1051] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:17.336064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596d230-540b-4398-9ebc-12e590d51771 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:17.384494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:17.396265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143aeed-4dae-4a06-8383-1c5722ce0b66 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:17.434355 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e6ded-c5a2-4a72-bac0-df89fa4d3636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:17.435180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e6ded-c5a2-4a72-bac0-df89fa4d3636 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.435466 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e6ded-c5a2-4a72-bac0-df89fa4d3636 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.435746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e6ded-c5a2-4a72-bac0-df89fa4d3636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.438933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:17.443567 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596d230-540b-4398-9ebc-12e590d51771 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:17.452342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e6ded-c5a2-4a72-bac0-df89fa4d3636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:17.454121 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e6ded-c5a2-4a72-bac0-df89fa4d3636 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-221e6ded-c5a2-4a72-bac0-df89fa4d3636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:46:17.471755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:17.472707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:17.473976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:46:17.473976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:46:17.473976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:17.473976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:17.473976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 521/1052] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:17.479168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e6ded-c5a2-4a72-bac0-df89fa4d3636 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:17.483957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794f42-7a54-47ab-aef9-afc706173a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:17.484898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794f42-7a54-47ab-aef9-afc706173a9f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.485177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794f42-7a54-47ab-aef9-afc706173a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.485452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794f42-7a54-47ab-aef9-afc706173a9f None admin] PATH_INFO: `/v3/services/image` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.486115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2794f42-7a54-47ab-aef9-afc706173a9f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:17.487606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2794f42-7a54-47ab-aef9-afc706173a9f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-b2794f42-7a54-47ab-aef9-afc706173a9f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:17.500776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:17.501932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:17.502832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 532/1053] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Jun 29 09:46:17 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:17.508125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794f42-7a54-47ab-aef9-afc706173a9f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:17.594082 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e6ded-c5a2-4a72-bac0-df89fa4d3636 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:17.625915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794f42-7a54-47ab-aef9-afc706173a9f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:17.636751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:17.637016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d200585-d4d6-4ea9-b6b9-daa3e7d2b33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:17.638299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d200585-d4d6-4ea9-b6b9-daa3e7d2b33c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.638693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d200585-d4d6-4ea9-b6b9-daa3e7d2b33c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.638950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d200585-d4d6-4ea9-b6b9-daa3e7d2b33c None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.639824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d200585-d4d6-4ea9-b6b9-daa3e7d2b33c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:17.640093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d200585-d4d6-4ea9-b6b9-daa3e7d2b33c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:17.641733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d200585-d4d6-4ea9-b6b9-daa3e7d2b33c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:17.660904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 522/1054] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:17.661759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d200585-d4d6-4ea9-b6b9-daa3e7d2b33c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:17.662331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5e255-0f78-43be-8cdd-8b6b95f55a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:17.663098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5e255-0f78-43be-8cdd-8b6b95f55a55 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.663383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5e255-0f78-43be-8cdd-8b6b95f55a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.663661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5e255-0f78-43be-8cdd-8b6b95f55a55 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.664435 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5e255-0f78-43be-8cdd-8b6b95f55a55 None admin] RBAC: Adding query filter params (name=image) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:17.664749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5e255-0f78-43be-8cdd-8b6b95f55a55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:17.666176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5e255-0f78-43be-8cdd-8b6b95f55a55 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:17.676375 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 533/1055] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:17.677282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5e255-0f78-43be-8cdd-8b6b95f55a55 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:17.715604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:17.769891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d200585-d4d6-4ea9-b6b9-daa3e7d2b33c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:17.797152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5e255-0f78-43be-8cdd-8b6b95f55a55 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:17.805319 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae64ecc-c6f7-47d8-8cf9-01ec46782253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:17.806144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae64ecc-c6f7-47d8-8cf9-01ec46782253 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.806426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae64ecc-c6f7-47d8-8cf9-01ec46782253 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.806732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae64ecc-c6f7-47d8-8cf9-01ec46782253 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.814387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae64ecc-c6f7-47d8-8cf9-01ec46782253 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:17.816216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae64ecc-c6f7-47d8-8cf9-01ec46782253 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-7ae64ecc-c6f7-47d8-8cf9-01ec46782253 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:46:17.831233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:17.832279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:17.833286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:17.833286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:17.833286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:46:17.833286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:46:17.833286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:46:17.833286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:46:17.833286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:46:17.833286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:46:17.833286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:46:17.833286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:17.833286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 523/1056] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 09:46:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:17.838136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae64ecc-c6f7-47d8-8cf9-01ec46782253 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:17.839428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7742d58-0422-4861-89ed-191692c3516d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:17.840131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7742d58-0422-4861-89ed-191692c3516d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.840407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7742d58-0422-4861-89ed-191692c3516d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.840679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7742d58-0422-4861-89ed-191692c3516d None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.841431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7742d58-0422-4861-89ed-191692c3516d None admin] RBAC: Adding query filter params (type=image) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:17.841834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7742d58-0422-4861-89ed-191692c3516d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:17.843221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7742d58-0422-4861-89ed-191692c3516d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:17.854066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 534/1057] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:17.855544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7742d58-0422-4861-89ed-191692c3516d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:17.902315 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:17.950773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae64ecc-c6f7-47d8-8cf9-01ec46782253 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:17.961141 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:17.967436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7742d58-0422-4861-89ed-191692c3516d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:17.989914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d2414-bcf8-493c-b974-f05c2087243f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:17.990620 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d2414-bcf8-493c-b974-f05c2087243f None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:17.990902 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d2414-bcf8-493c-b974-f05c2087243f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:17.991173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d2414-bcf8-493c-b974-f05c2087243f None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:17.992006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d2414-bcf8-493c-b974-f05c2087243f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:17.992309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d2414-bcf8-493c-b974-f05c2087243f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:17.993749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d2414-bcf8-493c-b974-f05c2087243f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:18.003065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85079e8-77d0-474c-8b0f-fa4951e15e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:18.003736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85079e8-77d0-474c-8b0f-fa4951e15e0f None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:18.004012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85079e8-77d0-474c-8b0f-fa4951e15e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:18.004284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85079e8-77d0-474c-8b0f-fa4951e15e0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:18.005022 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85079e8-77d0-474c-8b0f-fa4951e15e0f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:18.006401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85079e8-77d0-474c-8b0f-fa4951e15e0f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:18.012439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 524/1058] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 09:46:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:18.017999 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d2414-bcf8-493c-b974-f05c2087243f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:18.031251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 535/1059] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 09:46:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:46:18.138595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d2414-bcf8-493c-b974-f05c2087243f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:18.186727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e252f7-e6e9-4f07-a0ed-b96c89c36ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:18.187545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e252f7-e6e9-4f07-a0ed-b96c89c36ecd None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:18.187894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e252f7-e6e9-4f07-a0ed-b96c89c36ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:18.188167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e252f7-e6e9-4f07-a0ed-b96c89c36ecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:18.199441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e252f7-e6e9-4f07-a0ed-b96c89c36ecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:18.200901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e252f7-e6e9-4f07-a0ed-b96c89c36ecd None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-74e252f7-e6e9-4f07-a0ed-b96c89c36ecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:46:18.220234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:18.221326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:18.222381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:46:18.222381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:46:18.222381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:18.222381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:18.222381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 525/1060] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:18.229020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85079e8-77d0-474c-8b0f-fa4951e15e0f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:18.331284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:18.340904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85079e8-77d0-474c-8b0f-fa4951e15e0f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:18.376389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582ea41-a906-41d2-b44a-03ea6ab7a52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:18.377065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582ea41-a906-41d2-b44a-03ea6ab7a52a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:18.377341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582ea41-a906-41d2-b44a-03ea6ab7a52a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:18.377610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582ea41-a906-41d2-b44a-03ea6ab7a52a None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:18.378337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582ea41-a906-41d2-b44a-03ea6ab7a52a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:18.378684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582ea41-a906-41d2-b44a-03ea6ab7a52a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:18.380068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582ea41-a906-41d2-b44a-03ea6ab7a52a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:18.392789 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 536/1061] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:18.398455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e252f7-e6e9-4f07-a0ed-b96c89c36ecd None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:18.414457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:18.498955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e252f7-e6e9-4f07-a0ed-b96c89c36ecd None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:18.535901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2cc6b5-cf37-4066-8c90-723203812721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:18.536707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cc6b5-cf37-4066-8c90-723203812721 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:18.536983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cc6b5-cf37-4066-8c90-723203812721 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:18.537251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cc6b5-cf37-4066-8c90-723203812721 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:18.544452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2cc6b5-cf37-4066-8c90-723203812721 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:18.545901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2cc6b5-cf37-4066-8c90-723203812721 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-8f2cc6b5-cf37-4066-8c90-723203812721 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:46:18.559864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:18.560788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:18.561692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:46:18.561692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:46:18.561692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:46:18.561692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:46:18.561692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:46:18.561692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:46:18.561692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:18.561692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:18.566320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 526/1062] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:46:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:18.567389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582ea41-a906-41d2-b44a-03ea6ab7a52a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:18.672635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582ea41-a906-41d2-b44a-03ea6ab7a52a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:18.708045 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b382e-bc27-4618-b26a-ba43a4df7c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:18.708721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b382e-bc27-4618-b26a-ba43a4df7c0b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:18.708999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b382e-bc27-4618-b26a-ba43a4df7c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:18.709272 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b382e-bc27-4618-b26a-ba43a4df7c0b None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:18.710063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b382e-bc27-4618-b26a-ba43a4df7c0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:18.710371 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b382e-bc27-4618-b26a-ba43a4df7c0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:18.711855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b382e-bc27-4618-b26a-ba43a4df7c0b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:18.724862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 537/1063] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:46:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:18.731022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2cc6b5-cf37-4066-8c90-723203812721 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:18.809859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:18.835846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2cc6b5-cf37-4066-8c90-723203812721 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:18.871331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd791c-09d3-4c3e-92b4-c36a41ca2621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:18.872223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd791c-09d3-4c3e-92b4-c36a41ca2621 None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:18.872499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd791c-09d3-4c3e-92b4-c36a41ca2621 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:18.872739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd791c-09d3-4c3e-92b4-c36a41ca2621 None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/8a5eea7f539048c48ed10e72dd933df9/roles/67c7f3afe8244828abe1191b2b97ae35` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:18.875140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd791c-09d3-4c3e-92b4-c36a41ca2621 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=8a5eea7f539048c48ed10e72dd933df9, role_id=67c7f3afe8244828abe1191b2b97ae35)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:18.876894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd791c-09d3-4c3e-92b4-c36a41ca2621 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:18.894674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 527/1064] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:46:18 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/8a5eea7f539048c48ed10e72dd933df9/roles/67c7f3afe8244828abe1191b2b97ae35 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:46:19.260390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8031a15-3c9b-4b95-b72a-1c7f5d7383c1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:19.260795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8031a15-3c9b-4b95-b72a-1c7f5d7383c1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:19.260968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8031a15-3c9b-4b95-b72a-1c7f5d7383c1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:19.262527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 538/1065] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:19.267992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34d46c-1c38-444f-a82c-a6914423105a None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:19.270154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34d46c-1c38-444f-a82c-a6914423105a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:19.270154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34d46c-1c38-444f-a82c-a6914423105a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:19.285790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-ec34d46c-1c38-444f-a82c-a6914423105a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:19.287031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ec34d46c-1c38-444f-a82c-a6914423105a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:19.367290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 528/1066] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:19.390813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8031a15-3c9b-4b95-b72a-1c7f5d7383c1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:19.409506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:19.506264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8031a15-3c9b-4b95-b72a-1c7f5d7383c1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:19.541711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ad63e-e017-41b0-869d-0fc3a54c92b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:19.542436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ad63e-e017-41b0-869d-0fc3a54c92b0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:19.542745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ad63e-e017-41b0-869d-0fc3a54c92b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:19.543027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ad63e-e017-41b0-869d-0fc3a54c92b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:19.553582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ad63e-e017-41b0-869d-0fc3a54c92b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-383ad63e-e017-41b0-869d-0fc3a54c92b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:46:19.561208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:19.562192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:19.568436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 539/1067] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:46:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:19.570638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34d46c-1c38-444f-a82c-a6914423105a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:19.637135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:19.683875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34d46c-1c38-444f-a82c-a6914423105a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:19.725361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf59955-849e-4b10-8281-64ec24b49b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:19.726262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf59955-849e-4b10-8281-64ec24b49b08 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:19.726714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf59955-849e-4b10-8281-64ec24b49b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:19.726823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf59955-849e-4b10-8281-64ec24b49b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:19.727685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf59955-849e-4b10-8281-64ec24b49b08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:19.728028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf59955-849e-4b10-8281-64ec24b49b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:19.729591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf59955-849e-4b10-8281-64ec24b49b08 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:19.743539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 529/1068] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:46:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:19.745582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ad63e-e017-41b0-869d-0fc3a54c92b0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:19.825402 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:19.851557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ad63e-e017-41b0-869d-0fc3a54c92b0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:19.885867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e5b5c-55f5-433d-b24b-0f26c4f65be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:19.886797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5b5c-55f5-433d-b24b-0f26c4f65be5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:19.887081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5b5c-55f5-433d-b24b-0f26c4f65be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:19.887352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5b5c-55f5-433d-b24b-0f26c4f65be5 None admin] PATH_INFO: `/v3/users/617566cf4c0542828dae861562c72eb4` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:19.901389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e5b5c-55f5-433d-b24b-0f26c4f65be5 None admin] RBAC: Authorizing `identity:get_user(user_id=617566cf4c0542828dae861562c72eb4)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:19.902879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e5b5c-55f5-433d-b24b-0f26c4f65be5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:19.904786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 540/1069] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:19 2022] GET /identity/v3/users/617566cf4c0542828dae861562c72eb4 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:19.910708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf59955-849e-4b10-8281-64ec24b49b08 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:20.016985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf59955-849e-4b10-8281-64ec24b49b08 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:20.051089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:20.064229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855daee-28df-4202-8f8e-356fd325beaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:20.064895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855daee-28df-4202-8f8e-356fd325beaf None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:20.065170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855daee-28df-4202-8f8e-356fd325beaf None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:20.065435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855daee-28df-4202-8f8e-356fd325beaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:20.082162 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3f5c6-aec8-4e75-8ce7-79b18c930c0c None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:20.082463 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3f5c6-aec8-4e75-8ce7-79b18c930c0c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:20.082657 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3f5c6-aec8-4e75-8ce7-79b18c930c0c None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:20.084305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 541/1070] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:46:20.089035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d6893-ae8d-4c74-9446-510cf7358699 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:20.089588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d6893-ae8d-4c74-9446-510cf7358699 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:20.089588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d6893-ae8d-4c74-9446-510cf7358699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:20.107706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-601d6893-ae8d-4c74-9446-510cf7358699 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:20.108943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-601d6893-ae8d-4c74-9446-510cf7358699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:20.140376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855daee-28df-4202-8f8e-356fd325beaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:20.142517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855daee-28df-4202-8f8e-356fd325beaf None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:20.180701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 542/1071] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:20.202794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d6893-ae8d-4c74-9446-510cf7358699 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:20.253569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 530/1072] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:46:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:46:20.256060 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855daee-28df-4202-8f8e-356fd325beaf None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:20.275145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:20.288916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:20.318976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d6893-ae8d-4c74-9446-510cf7358699 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:20.355103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6807-6962-4cbb-ac82-facf684c1b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:20.355658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6807-6962-4cbb-ac82-facf684c1b69 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:20.355901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6807-6962-4cbb-ac82-facf684c1b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:20.356175 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6807-6962-4cbb-ac82-facf684c1b69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:20.374274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855daee-28df-4202-8f8e-356fd325beaf None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:20.413660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c13b30-363a-4382-ab01-11fc3406388d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:20.414555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c13b30-363a-4382-ab01-11fc3406388d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:20.414868 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c13b30-363a-4382-ab01-11fc3406388d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:20.415136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c13b30-363a-4382-ab01-11fc3406388d None admin] PATH_INFO: `/v3/projects/aee657be89e3474aaf6d4c76a77d4152` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:20.416448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c13b30-363a-4382-ab01-11fc3406388d None admin] RBAC: Authorizing `identity:get_project(project_id=aee657be89e3474aaf6d4c76a77d4152)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:20.417995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c13b30-363a-4382-ab01-11fc3406388d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:20.420053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 531/1073] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:46:20 2022] GET /identity/v3/projects/aee657be89e3474aaf6d4c76a77d4152 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:20.427155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c13b30-363a-4382-ab01-11fc3406388d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:20.434185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc6807-6962-4cbb-ac82-facf684c1b69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:20.435733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc6807-6962-4cbb-ac82-facf684c1b69 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:20.507268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:20.540095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:20.549843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c13b30-363a-4382-ab01-11fc3406388d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:20.551380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 543/1074] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:46:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:46:20.557214 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6807-6962-4cbb-ac82-facf684c1b69 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:20.591484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca511d2-22c8-44c1-a235-28a98679ed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:20.592803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca511d2-22c8-44c1-a235-28a98679ed3a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:20.593444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca511d2-22c8-44c1-a235-28a98679ed3a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:20.594491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca511d2-22c8-44c1-a235-28a98679ed3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:20.595889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca511d2-22c8-44c1-a235-28a98679ed3a None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=aee657be89e3474aaf6d4c76a77d4152, user.id=617566cf4c0542828dae861562c72eb4) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:20.596411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca511d2-22c8-44c1-a235-28a98679ed3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:20.599194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca511d2-22c8-44c1-a235-28a98679ed3a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:20.610693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 532/1075] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:46:20 2022] GET /identity/v3/role_assignments?user.id=617566cf4c0542828dae861562c72eb4&scope.project.id=aee657be89e3474aaf6d4c76a77d4152&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:20.702749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6807-6962-4cbb-ac82-facf684c1b69 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:20.749376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab261cf8-a463-445f-b103-e202b2726fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:20.750363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab261cf8-a463-445f-b103-e202b2726fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:20.750900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab261cf8-a463-445f-b103-e202b2726fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:20.751304 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab261cf8-a463-445f-b103-e202b2726fb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:20.763748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab261cf8-a463-445f-b103-e202b2726fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:20.765314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab261cf8-a463-445f-b103-e202b2726fb0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-ab261cf8-a463-445f-b103-e202b2726fb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:46:20.785081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:20.786043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:20.786975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:46:20.786975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:46:20.786975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:20.786975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:20.786975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 544/1076] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:20.792631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca511d2-22c8-44c1-a235-28a98679ed3a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:20.914407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca511d2-22c8-44c1-a235-28a98679ed3a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:20.955748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed957f9-e16a-4b24-88e8-df8e99f35f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:20.956489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed957f9-e16a-4b24-88e8-df8e99f35f73 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:20.956769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed957f9-e16a-4b24-88e8-df8e99f35f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:20.957038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed957f9-e16a-4b24-88e8-df8e99f35f73 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:20.957770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed957f9-e16a-4b24-88e8-df8e99f35f73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:20.958072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed957f9-e16a-4b24-88e8-df8e99f35f73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:20.959474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed957f9-e16a-4b24-88e8-df8e99f35f73 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:20.979048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 533/1077] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:21.025749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42492d38-713b-4d96-b905-4a346032341e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:21.026055 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42492d38-713b-4d96-b905-4a346032341e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:21.026302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42492d38-713b-4d96-b905-4a346032341e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:21.027787 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 545/1078] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:21.033843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e7281-1fa6-4e68-8566-c6966a6fc0e4 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:21.034178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e7281-1fa6-4e68-8566-c6966a6fc0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:21.034449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e7281-1fa6-4e68-8566-c6966a6fc0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:21.054504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a84e7281-1fa6-4e68-8566-c6966a6fc0e4 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:21.055961 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a84e7281-1fa6-4e68-8566-c6966a6fc0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:21.072379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:21.145543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 534/1079] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:21.170026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42492d38-713b-4d96-b905-4a346032341e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:21.189164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:21.310774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42492d38-713b-4d96-b905-4a346032341e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:21.350953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935f11f-a1df-44e9-9626-21a20d3329eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:21.351819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935f11f-a1df-44e9-9626-21a20d3329eb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:21.352100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935f11f-a1df-44e9-9626-21a20d3329eb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:21.352370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935f11f-a1df-44e9-9626-21a20d3329eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:21.365098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935f11f-a1df-44e9-9626-21a20d3329eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-8935f11f-a1df-44e9-9626-21a20d3329eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:46:21.373266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:21.374431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:21.375893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 546/1080] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 09:46:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:21.380855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e7281-1fa6-4e68-8566-c6966a6fc0e4 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:21.492540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e7281-1fa6-4e68-8566-c6966a6fc0e4 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:21.530680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7e58d-b95c-4f1a-b4b2-59cec8c7c243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:21.531389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7e58d-b95c-4f1a-b4b2-59cec8c7c243 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:21.531705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7e58d-b95c-4f1a-b4b2-59cec8c7c243 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:21.531966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7e58d-b95c-4f1a-b4b2-59cec8c7c243 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:21.532720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7e58d-b95c-4f1a-b4b2-59cec8c7c243 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:21.533031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7e58d-b95c-4f1a-b4b2-59cec8c7c243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:21.534431 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7e58d-b95c-4f1a-b4b2-59cec8c7c243 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:21.546167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 535/1081] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 09:46:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:21.553614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935f11f-a1df-44e9-9626-21a20d3329eb None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:21.650132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:21.681819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935f11f-a1df-44e9-9626-21a20d3329eb None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:21.732016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d01ac-1933-4536-b72f-52ddb4121eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:21.732935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d01ac-1933-4536-b72f-52ddb4121eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:21.733215 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d01ac-1933-4536-b72f-52ddb4121eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:21.733484 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d01ac-1933-4536-b72f-52ddb4121eb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:21.746807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d01ac-1933-4536-b72f-52ddb4121eb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:21.748228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d01ac-1933-4536-b72f-52ddb4121eb2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-3f6d01ac-1933-4536-b72f-52ddb4121eb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:46:21.766867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:21.768405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:21.769353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:46:21.769353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:46:21.769353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:21.769353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:21.769353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 547/1082] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:21.773987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7e58d-b95c-4f1a-b4b2-59cec8c7c243 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:21.787907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:21.882405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7e58d-b95c-4f1a-b4b2-59cec8c7c243 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:21.919559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61239921-aa55-4265-9ede-f94e500a2f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:21.920246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61239921-aa55-4265-9ede-f94e500a2f36 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:21.920529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61239921-aa55-4265-9ede-f94e500a2f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:21.920809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61239921-aa55-4265-9ede-f94e500a2f36 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:21.921540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61239921-aa55-4265-9ede-f94e500a2f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:21.921857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61239921-aa55-4265-9ede-f94e500a2f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:21.926380 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61239921-aa55-4265-9ede-f94e500a2f36 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:21.944641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 536/1083] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:21.950516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d01ac-1933-4536-b72f-52ddb4121eb2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:22.071569 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d01ac-1933-4536-b72f-52ddb4121eb2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:22.102736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:22.113600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43542fb-7f2c-40cd-8207-01b31fbd901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:22.114410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43542fb-7f2c-40cd-8207-01b31fbd901a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:22.114723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43542fb-7f2c-40cd-8207-01b31fbd901a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:22.115026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43542fb-7f2c-40cd-8207-01b31fbd901a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:22.123014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43542fb-7f2c-40cd-8207-01b31fbd901a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:22.124473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43542fb-7f2c-40cd-8207-01b31fbd901a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-c43542fb-7f2c-40cd-8207-01b31fbd901a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:46:22.139523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:22.140604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:22.141626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:22.141626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:22.141626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:46:22.141626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:46:22.141626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:46:22.141626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:46:22.141626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:46:22.141626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:46:22.141626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 09:46:22.141626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:22.141626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 548/1084] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 09:46:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:22.148551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61239921-aa55-4265-9ede-f94e500a2f36 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:22.264694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:22.300430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61239921-aa55-4265-9ede-f94e500a2f36 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:22.337659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf449b8-05b5-40ed-8e38-f47413efc9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:22.338228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf449b8-05b5-40ed-8e38-f47413efc9df None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:22.338510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf449b8-05b5-40ed-8e38-f47413efc9df None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:22.338817 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf449b8-05b5-40ed-8e38-f47413efc9df None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:22.339666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf449b8-05b5-40ed-8e38-f47413efc9df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:22.340018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf449b8-05b5-40ed-8e38-f47413efc9df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:22.341446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf449b8-05b5-40ed-8e38-f47413efc9df None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:22.364719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 537/1085] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 09:46:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:22.370683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43542fb-7f2c-40cd-8207-01b31fbd901a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:22.480235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43542fb-7f2c-40cd-8207-01b31fbd901a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:22.517417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57dbce-e789-4fb3-82de-0cbc23f22823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:22.518062 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57dbce-e789-4fb3-82de-0cbc23f22823 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:22.518343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57dbce-e789-4fb3-82de-0cbc23f22823 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:22.518644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57dbce-e789-4fb3-82de-0cbc23f22823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:22.577048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:22.591549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57dbce-e789-4fb3-82de-0cbc23f22823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:22.592295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57dbce-e789-4fb3-82de-0cbc23f22823 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:22.646790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8c688-5ea5-475d-9bd3-62ef4d63a3ad None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:22.647043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8c688-5ea5-475d-9bd3-62ef4d63a3ad None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:22.647291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8c688-5ea5-475d-9bd3-62ef4d63a3ad None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:22.648868 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 538/1086] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:46:22.654010 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf3170-d3ee-4c52-867b-1799c72dfbae None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:22.654295 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf3170-d3ee-4c52-867b-1799c72dfbae None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:22.654589 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf3170-d3ee-4c52-867b-1799c72dfbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:22.672676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-47bf3170-d3ee-4c52-867b-1799c72dfbae None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:22.674002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-47bf3170-d3ee-4c52-867b-1799c72dfbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:22.700558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 549/1087] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:46:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:46:22.706548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57dbce-e789-4fb3-82de-0cbc23f22823 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:22.752697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 539/1088] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:22.776009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf3170-d3ee-4c52-867b-1799c72dfbae None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:22.795509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:22.812168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57dbce-e789-4fb3-82de-0cbc23f22823 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:22.840076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:22.863885 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a603cc-5893-4a29-ac5f-7dc009041764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:22.864607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a603cc-5893-4a29-ac5f-7dc009041764 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:22.864883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a603cc-5893-4a29-ac5f-7dc009041764 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:22.865151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a603cc-5893-4a29-ac5f-7dc009041764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:22.875812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a603cc-5893-4a29-ac5f-7dc009041764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:22.877303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a603cc-5893-4a29-ac5f-7dc009041764 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:22.884449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf3170-d3ee-4c52-867b-1799c72dfbae None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-26a603cc-5893-4a29-ac5f-7dc009041764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:46:22.895012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:22.895961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:22.896864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:46:22.896864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:46:22.896864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:46:22.896864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:22.896864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 550/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:22.902071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a603cc-5893-4a29-ac5f-7dc009041764 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:22.924160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd1be4-d3b4-4f93-99b0-e551845b5a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:22.924989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd1be4-d3b4-4f93-99b0-e551845b5a64 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:22.925273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd1be4-d3b4-4f93-99b0-e551845b5a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:22.925539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd1be4-d3b4-4f93-99b0-e551845b5a64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:22.936936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd1be4-d3b4-4f93-99b0-e551845b5a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-edbd1be4-d3b4-4f93-99b0-e551845b5a64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:46:22.944737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:22.945762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:22.946701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 540/1090] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:46:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:22.952317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd1be4-d3b4-4f93-99b0-e551845b5a64 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:23.008476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a603cc-5893-4a29-ac5f-7dc009041764 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:23.045964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151887d2-2bdc-4c07-b9b9-faee43c96535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:23.046728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151887d2-2bdc-4c07-b9b9-faee43c96535 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:23.047130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151887d2-2bdc-4c07-b9b9-faee43c96535 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:23.047434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151887d2-2bdc-4c07-b9b9-faee43c96535 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:23.048180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151887d2-2bdc-4c07-b9b9-faee43c96535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:23.048489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151887d2-2bdc-4c07-b9b9-faee43c96535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:23.049865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151887d2-2bdc-4c07-b9b9-faee43c96535 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:23.056948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:23.057280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd1be4-d3b4-4f93-99b0-e551845b5a64 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:23.063204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 551/1091] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:23.068935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151887d2-2bdc-4c07-b9b9-faee43c96535 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:23.076350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:23.095150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02857b57-094a-49fc-a5d2-c35e95d3486a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:23.095828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02857b57-094a-49fc-a5d2-c35e95d3486a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:23.096109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02857b57-094a-49fc-a5d2-c35e95d3486a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:23.096379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02857b57-094a-49fc-a5d2-c35e95d3486a None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:23.097133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02857b57-094a-49fc-a5d2-c35e95d3486a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:23.097439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02857b57-094a-49fc-a5d2-c35e95d3486a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:23.099202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02857b57-094a-49fc-a5d2-c35e95d3486a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:23.108121 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 541/1092] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:46:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:23.114741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02857b57-094a-49fc-a5d2-c35e95d3486a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:23.173939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151887d2-2bdc-4c07-b9b9-faee43c96535 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:23.212108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c2286-2296-4cdc-9ec6-241c633c4372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:23.212922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c2286-2296-4cdc-9ec6-241c633c4372 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:23.213164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c2286-2296-4cdc-9ec6-241c633c4372 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:23.213436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c2286-2296-4cdc-9ec6-241c633c4372 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:23.221065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c2286-2296-4cdc-9ec6-241c633c4372 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:23.222546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c2286-2296-4cdc-9ec6-241c633c4372 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:23.227995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02857b57-094a-49fc-a5d2-c35e95d3486a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-3f3c2286-2296-4cdc-9ec6-241c633c4372 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:46:23.238780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:23.239935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:23.240857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:46:23.240857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:46:23.240857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:46:23.240857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:46:23.240857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:46:23.240857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:46:23.240857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:46:23.240857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:23.240857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 552/1093] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:46:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:46:23.246707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c2286-2296-4cdc-9ec6-241c633c4372 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:23.265540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13482282-0095-4cef-a432-ef02697bfbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:23.266787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482282-0095-4cef-a432-ef02697bfbff None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:23.267144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482282-0095-4cef-a432-ef02697bfbff None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:23.267419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482282-0095-4cef-a432-ef02697bfbff None admin] PATH_INFO: `/v3/users/617566cf4c0542828dae861562c72eb4` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:23.268771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13482282-0095-4cef-a432-ef02697bfbff None admin] RBAC: Authorizing `identity:get_user(user_id=617566cf4c0542828dae861562c72eb4)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:23.270248 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13482282-0095-4cef-a432-ef02697bfbff None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:23.272448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 542/1094] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:23 2022] GET /identity/v3/users/617566cf4c0542828dae861562c72eb4 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:23.278410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13482282-0095-4cef-a432-ef02697bfbff None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:23.314358 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:23.333708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:23.361347 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c2286-2296-4cdc-9ec6-241c633c4372 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:23.390313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13482282-0095-4cef-a432-ef02697bfbff None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:23.397951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eba5fe-df85-42db-9047-ae083ca7cc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:23.398679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eba5fe-df85-42db-9047-ae083ca7cc21 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:23.398966 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eba5fe-df85-42db-9047-ae083ca7cc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:23.399239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eba5fe-df85-42db-9047-ae083ca7cc21 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:23.400037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eba5fe-df85-42db-9047-ae083ca7cc21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:23.400356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eba5fe-df85-42db-9047-ae083ca7cc21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:23.401822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eba5fe-df85-42db-9047-ae083ca7cc21 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:23.414366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 553/1095] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:46:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:23.420547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eba5fe-df85-42db-9047-ae083ca7cc21 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:23.426251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed1be5-dc78-4e45-bfa9-ff069ebe2c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:23.427193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1be5-dc78-4e45-bfa9-ff069ebe2c2d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:23.427469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1be5-dc78-4e45-bfa9-ff069ebe2c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:23.427740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1be5-dc78-4e45-bfa9-ff069ebe2c2d None admin] PATH_INFO: `/v3/projects/aee657be89e3474aaf6d4c76a77d4152` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:23.429060 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed1be5-dc78-4e45-bfa9-ff069ebe2c2d None admin] RBAC: Authorizing `identity:get_project(project_id=aee657be89e3474aaf6d4c76a77d4152)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:23.430528 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed1be5-dc78-4e45-bfa9-ff069ebe2c2d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:23.432359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 543/1096] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:46:23 2022] GET /identity/v3/projects/aee657be89e3474aaf6d4c76a77d4152 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:23.438335 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed1be5-dc78-4e45-bfa9-ff069ebe2c2d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:23.530222 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eba5fe-df85-42db-9047-ae083ca7cc21 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:23.540123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed1be5-dc78-4e45-bfa9-ff069ebe2c2d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:23.547739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:23.556678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:23.567090 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c88e8d-3a78-4bf4-8168-8cb33db42792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:23.567757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c88e8d-3a78-4bf4-8168-8cb33db42792 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:23.569211 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c88e8d-3a78-4bf4-8168-8cb33db42792 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:23.569479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c88e8d-3a78-4bf4-8168-8cb33db42792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:23.570313 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c88e8d-3a78-4bf4-8168-8cb33db42792 None admin] RBAC: Adding query filter params (role.id=67c7f3afe8244828abe1191b2b97ae35, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=8a5eea7f539048c48ed10e72dd933df9) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:23.570624 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c88e8d-3a78-4bf4-8168-8cb33db42792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:23.572089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c88e8d-3a78-4bf4-8168-8cb33db42792 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:23.576633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5925ff-3568-41e0-b831-993cc8e5a61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:23.577536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5925ff-3568-41e0-b831-993cc8e5a61e None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:23.577818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5925ff-3568-41e0-b831-993cc8e5a61e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:23.578094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5925ff-3568-41e0-b831-993cc8e5a61e None admin] PATH_INFO: `/v3/projects/aee657be89e3474aaf6d4c76a77d4152/users/617566cf4c0542828dae861562c72eb4/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:23.582779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 554/1097] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:46:23 2022] GET /identity/v3/role_assignments?user.id=8a5eea7f539048c48ed10e72dd933df9&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=67c7f3afe8244828abe1191b2b97ae35 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:23.582909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5925ff-3568-41e0-b831-993cc8e5a61e None admin] RBAC: Authorizing `identity:create_grant(project_id=aee657be89e3474aaf6d4c76a77d4152, user_id=617566cf4c0542828dae861562c72eb4, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:23.582909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5925ff-3568-41e0-b831-993cc8e5a61e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:23.597252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 544/1098] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:46:23 2022] PUT /identity/v3/projects/aee657be89e3474aaf6d4c76a77d4152/users/617566cf4c0542828dae861562c72eb4/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:46:25.759206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44401bbc-58b1-4220-b8f7-4fcc151a2a8b None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:25.759773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44401bbc-58b1-4220-b8f7-4fcc151a2a8b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:25.759773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44401bbc-58b1-4220-b8f7-4fcc151a2a8b None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:25.761107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 555/1099] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:25.766671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b992ece-39c2-4cb5-afb2-7889f8ff05b5 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:25.767089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b992ece-39c2-4cb5-afb2-7889f8ff05b5 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:25.767291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b992ece-39c2-4cb5-afb2-7889f8ff05b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:25.785914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-7b992ece-39c2-4cb5-afb2-7889f8ff05b5 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:25.787153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7b992ece-39c2-4cb5-afb2-7889f8ff05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:25.865633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 545/1100] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:25.889876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44401bbc-58b1-4220-b8f7-4fcc151a2a8b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:25.997731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44401bbc-58b1-4220-b8f7-4fcc151a2a8b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:26.031371 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55745830-8d7e-4400-bb3a-a016e3165978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:26.032095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55745830-8d7e-4400-bb3a-a016e3165978 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:26.032374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55745830-8d7e-4400-bb3a-a016e3165978 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:26.032646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55745830-8d7e-4400-bb3a-a016e3165978 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:26.045594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55745830-8d7e-4400-bb3a-a016e3165978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-55745830-8d7e-4400-bb3a-a016e3165978 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:46:26.053498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:26.055602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:26.057961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 556/1101] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:46:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:26.062553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b992ece-39c2-4cb5-afb2-7889f8ff05b5 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:26.169696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b992ece-39c2-4cb5-afb2-7889f8ff05b5 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:26.177655 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:26.207343 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb49355-876c-4553-b429-9532a3d1bb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:26.208018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb49355-876c-4553-b429-9532a3d1bb72 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:26.208301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb49355-876c-4553-b429-9532a3d1bb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:26.208576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb49355-876c-4553-b429-9532a3d1bb72 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:26.209326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb49355-876c-4553-b429-9532a3d1bb72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:26.209634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb49355-876c-4553-b429-9532a3d1bb72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:26.211052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb49355-876c-4553-b429-9532a3d1bb72 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:26.219729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 546/1102] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:46:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:26.226246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55745830-8d7e-4400-bb3a-a016e3165978 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:26.273310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:26.332368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55745830-8d7e-4400-bb3a-a016e3165978 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:26.367195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa398f60-71c7-49a5-a503-ee889e6c4998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:26.368077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa398f60-71c7-49a5-a503-ee889e6c4998 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:26.368353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa398f60-71c7-49a5-a503-ee889e6c4998 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:26.368622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa398f60-71c7-49a5-a503-ee889e6c4998 None admin] PATH_INFO: `/v3/users/617566cf4c0542828dae861562c72eb4` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:26.369850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa398f60-71c7-49a5-a503-ee889e6c4998 None admin] RBAC: Authorizing `identity:get_user(user_id=617566cf4c0542828dae861562c72eb4)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:26.371341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa398f60-71c7-49a5-a503-ee889e6c4998 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:26.373050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 557/1103] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:26 2022] GET /identity/v3/users/617566cf4c0542828dae861562c72eb4 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:26.379345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb49355-876c-4553-b429-9532a3d1bb72 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:26.485271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb49355-876c-4553-b429-9532a3d1bb72 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:26.522803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728d2c2-5cc5-46c3-a0a2-a514a03ef827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:26.523465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728d2c2-5cc5-46c3-a0a2-a514a03ef827 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:26.523746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728d2c2-5cc5-46c3-a0a2-a514a03ef827 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:26.524019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728d2c2-5cc5-46c3-a0a2-a514a03ef827 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:26.560335 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:26.596924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728d2c2-5cc5-46c3-a0a2-a514a03ef827 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:26.598695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728d2c2-5cc5-46c3-a0a2-a514a03ef827 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:26.700005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 547/1104] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:46:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:46:26.706183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa398f60-71c7-49a5-a503-ee889e6c4998 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:26.812490 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa398f60-71c7-49a5-a503-ee889e6c4998 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:26.823612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:26.849857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75084fc1-7ead-428d-a57d-94c312447755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:26.850630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75084fc1-7ead-428d-a57d-94c312447755 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:26.850934 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75084fc1-7ead-428d-a57d-94c312447755 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:26.851204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75084fc1-7ead-428d-a57d-94c312447755 None admin] PATH_INFO: `/v3/projects/aee657be89e3474aaf6d4c76a77d4152` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:26.852420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75084fc1-7ead-428d-a57d-94c312447755 None admin] RBAC: Authorizing `identity:get_project(project_id=aee657be89e3474aaf6d4c76a77d4152)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:26.853862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75084fc1-7ead-428d-a57d-94c312447755 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:26.855614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 558/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:46:26 2022] GET /identity/v3/projects/aee657be89e3474aaf6d4c76a77d4152 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:26.862455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728d2c2-5cc5-46c3-a0a2-a514a03ef827 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:26.930244 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:26.966893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728d2c2-5cc5-46c3-a0a2-a514a03ef827 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:27.003771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead710f-fc52-4e5b-b1cc-087e68d8d168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:27.004422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead710f-fc52-4e5b-b1cc-087e68d8d168 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:27.004701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead710f-fc52-4e5b-b1cc-087e68d8d168 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:27.004970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead710f-fc52-4e5b-b1cc-087e68d8d168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:27.005796 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead710f-fc52-4e5b-b1cc-087e68d8d168 None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=aee657be89e3474aaf6d4c76a77d4152, user.id=617566cf4c0542828dae861562c72eb4) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:27.006105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead710f-fc52-4e5b-b1cc-087e68d8d168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:27.007569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead710f-fc52-4e5b-b1cc-087e68d8d168 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:27.013746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 548/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:46:26 2022] GET /identity/v3/role_assignments?user.id=617566cf4c0542828dae861562c72eb4&scope.project.id=aee657be89e3474aaf6d4c76a77d4152&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:29.085838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e7dd-c0d7-4c52-b158-4a32b9188599 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:29.086404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e7dd-c0d7-4c52-b158-4a32b9188599 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:29.086404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e7dd-c0d7-4c52-b158-4a32b9188599 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:29.087803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 559/1107] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:29.093096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e711fa-357f-4ff3-89d1-6bf903cf9957 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:29.093540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e711fa-357f-4ff3-89d1-6bf903cf9957 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:29.093667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e711fa-357f-4ff3-89d1-6bf903cf9957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:29.111422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-57e711fa-357f-4ff3-89d1-6bf903cf9957 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:29.112607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-57e711fa-357f-4ff3-89d1-6bf903cf9957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:29.185265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 549/1108] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:29.210096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92e7dd-c0d7-4c52-b158-4a32b9188599 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:29.318095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92e7dd-c0d7-4c52-b158-4a32b9188599 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:29.354019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da253774-47c4-4401-9d35-f18cc477a4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:29.354845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da253774-47c4-4401-9d35-f18cc477a4ea None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:29.355121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da253774-47c4-4401-9d35-f18cc477a4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:29.355395 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da253774-47c4-4401-9d35-f18cc477a4ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:29.356618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da253774-47c4-4401-9d35-f18cc477a4ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:29.358071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da253774-47c4-4401-9d35-f18cc477a4ea None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:29.359911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 560/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:29.365949 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e711fa-357f-4ff3-89d1-6bf903cf9957 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:29.432933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:29.474710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e711fa-357f-4ff3-89d1-6bf903cf9957 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:29.510929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01ff79-437d-4643-ad7c-fd564dd1cd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:29.511611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01ff79-437d-4643-ad7c-fd564dd1cd42 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:29.511893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01ff79-437d-4643-ad7c-fd564dd1cd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:29.512170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01ff79-437d-4643-ad7c-fd564dd1cd42 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:29.512888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01ff79-437d-4643-ad7c-fd564dd1cd42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:29.514336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01ff79-437d-4643-ad7c-fd564dd1cd42 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:29.535606 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 550/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:46:29 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:46:31.622531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63974bc-8a20-4bb2-9cc7-4eba5bb86807 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:31.623160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63974bc-8a20-4bb2-9cc7-4eba5bb86807 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:31.623160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63974bc-8a20-4bb2-9cc7-4eba5bb86807 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:31.624503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 561/1111] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:31.630003 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea04eff-9781-49a6-8069-38e899c3e984 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:31.630444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea04eff-9781-49a6-8069-38e899c3e984 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:31.630549 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea04eff-9781-49a6-8069-38e899c3e984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:31.649154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-5ea04eff-9781-49a6-8069-38e899c3e984 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:31.650360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5ea04eff-9781-49a6-8069-38e899c3e984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:31.721418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 551/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:31.746580 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63974bc-8a20-4bb2-9cc7-4eba5bb86807 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:31.798178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:31.856486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63974bc-8a20-4bb2-9cc7-4eba5bb86807 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:31.894931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9a37d-a0dd-4467-8783-fcee06766c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:31.895725 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9a37d-a0dd-4467-8783-fcee06766c16 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:31.896008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9a37d-a0dd-4467-8783-fcee06766c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:31.896279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9a37d-a0dd-4467-8783-fcee06766c16 None admin] PATH_INFO: `/v3/roles/f819d0f29adf41a49fa045dc3c031527` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:31.897856 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9a37d-a0dd-4467-8783-fcee06766c16 None admin] RBAC: Authorizing `identity:get_role(role_id=f819d0f29adf41a49fa045dc3c031527)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:31.899288 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9a37d-a0dd-4467-8783-fcee06766c16 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:31.900736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 562/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:31 2022] GET /identity/v3/roles/f819d0f29adf41a49fa045dc3c031527 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:31.908321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea04eff-9781-49a6-8069-38e899c3e984 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:31.965367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:32.025580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea04eff-9781-49a6-8069-38e899c3e984 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:32.068308 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333bba0-a648-4dcf-948e-826000163e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:32.069207 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333bba0-a648-4dcf-948e-826000163e80 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:32.069494 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333bba0-a648-4dcf-948e-826000163e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:32.069771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333bba0-a648-4dcf-948e-826000163e80 None admin] PATH_INFO: `/v3/users/63dc8a977d3c4d10be8e1cef85e85e5b` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:32.084705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333bba0-a648-4dcf-948e-826000163e80 None admin] RBAC: Authorizing `identity:get_user(user_id=63dc8a977d3c4d10be8e1cef85e85e5b)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:32.086167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333bba0-a648-4dcf-948e-826000163e80 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:32.087982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 552/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:31 2022] GET /identity/v3/users/63dc8a977d3c4d10be8e1cef85e85e5b => generated 286 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:32.094044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9a37d-a0dd-4467-8783-fcee06766c16 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:32.202022 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9a37d-a0dd-4467-8783-fcee06766c16 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:32.217187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:32.238642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebe9b0-7faf-4def-aab5-421518868437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:32.239296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebe9b0-7faf-4def-aab5-421518868437 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:32.239578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebe9b0-7faf-4def-aab5-421518868437 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:32.239851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebe9b0-7faf-4def-aab5-421518868437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:32.312749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebe9b0-7faf-4def-aab5-421518868437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:32.314257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebe9b0-7faf-4def-aab5-421518868437 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:32.420093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 563/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:46:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:46:32.426274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333bba0-a648-4dcf-948e-826000163e80 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:32.535697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333bba0-a648-4dcf-948e-826000163e80 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:32.543933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:32.574403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6325af2-c30d-46f6-88dc-096f83965658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:32.575332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6325af2-c30d-46f6-88dc-096f83965658 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:32.575601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6325af2-c30d-46f6-88dc-096f83965658 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:32.575876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6325af2-c30d-46f6-88dc-096f83965658 None admin] PATH_INFO: `/v3/projects/aee657be89e3474aaf6d4c76a77d4152` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:32.577146 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6325af2-c30d-46f6-88dc-096f83965658 None admin] RBAC: Authorizing `identity:get_project(project_id=aee657be89e3474aaf6d4c76a77d4152)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:32.578624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6325af2-c30d-46f6-88dc-096f83965658 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:32.580397 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 553/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:46:32 2022] GET /identity/v3/projects/aee657be89e3474aaf6d4c76a77d4152 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:32.586803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebe9b0-7faf-4def-aab5-421518868437 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:32.604271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:32.704728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebe9b0-7faf-4def-aab5-421518868437 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:32.742505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a195cbe-6d4f-4e0f-8c5e-ad08bb9478c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:32.743201 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a195cbe-6d4f-4e0f-8c5e-ad08bb9478c3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:32.743474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a195cbe-6d4f-4e0f-8c5e-ad08bb9478c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:32.743739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a195cbe-6d4f-4e0f-8c5e-ad08bb9478c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:32.744574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a195cbe-6d4f-4e0f-8c5e-ad08bb9478c3 None admin] RBAC: Adding query filter params (role.id=f819d0f29adf41a49fa045dc3c031527, scope.project.id=aee657be89e3474aaf6d4c76a77d4152, user.id=63dc8a977d3c4d10be8e1cef85e85e5b) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:32.744877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a195cbe-6d4f-4e0f-8c5e-ad08bb9478c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:32.746345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a195cbe-6d4f-4e0f-8c5e-ad08bb9478c3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:32.753392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 564/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:46:32 2022] GET /identity/v3/role_assignments?user.id=63dc8a977d3c4d10be8e1cef85e85e5b&scope.project.id=aee657be89e3474aaf6d4c76a77d4152&role.id=f819d0f29adf41a49fa045dc3c031527 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:34.821593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aff9e1-0079-4204-891a-f3ae40e35ae1 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:34.822217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aff9e1-0079-4204-891a-f3ae40e35ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:34.822217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aff9e1-0079-4204-891a-f3ae40e35ae1 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:34.823728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 554/1118] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:46:34.829788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6864c09-1649-4f0c-915e-c7da08534779 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:34.830362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6864c09-1649-4f0c-915e-c7da08534779 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:34.830362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6864c09-1649-4f0c-915e-c7da08534779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:34.849387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f6864c09-1649-4f0c-915e-c7da08534779 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:34.850762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f6864c09-1649-4f0c-915e-c7da08534779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:34.921354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 565/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:34.944801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aff9e1-0079-4204-891a-f3ae40e35ae1 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:35.069589 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aff9e1-0079-4204-891a-f3ae40e35ae1 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:35.111598 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a5e72-1df8-4235-8b3c-5e2be5dea86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:35.112376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a5e72-1df8-4235-8b3c-5e2be5dea86d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:35.112652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a5e72-1df8-4235-8b3c-5e2be5dea86d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:35.112917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a5e72-1df8-4235-8b3c-5e2be5dea86d None admin] PATH_INFO: `/v3/roles/f819d0f29adf41a49fa045dc3c031527` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:35.114531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a5e72-1df8-4235-8b3c-5e2be5dea86d None admin] RBAC: Authorizing `identity:get_role(role_id=f819d0f29adf41a49fa045dc3c031527)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:35.115959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a5e72-1df8-4235-8b3c-5e2be5dea86d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:35.117417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 555/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:34 2022] GET /identity/v3/roles/f819d0f29adf41a49fa045dc3c031527 => generated 223 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:35.123779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6864c09-1649-4f0c-915e-c7da08534779 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:35.134697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:35.235327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6864c09-1649-4f0c-915e-c7da08534779 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:35.273232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed7468-e557-451e-aaf3-1986e21e5c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:35.274172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed7468-e557-451e-aaf3-1986e21e5c72 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:35.274499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed7468-e557-451e-aaf3-1986e21e5c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:35.274855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed7468-e557-451e-aaf3-1986e21e5c72 None admin] PATH_INFO: `/v3/users/63dc8a977d3c4d10be8e1cef85e85e5b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:35.276128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed7468-e557-451e-aaf3-1986e21e5c72 None admin] RBAC: Authorizing `identity:get_user(user_id=63dc8a977d3c4d10be8e1cef85e85e5b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:35.277628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed7468-e557-451e-aaf3-1986e21e5c72 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:35.279376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 566/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:35 2022] GET /identity/v3/users/63dc8a977d3c4d10be8e1cef85e85e5b => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:35.286461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a5e72-1df8-4235-8b3c-5e2be5dea86d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:35.333286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:35.397837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a5e72-1df8-4235-8b3c-5e2be5dea86d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:35.445122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c97c53-51d5-420c-9ab9-3eb387788042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:35.446006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c97c53-51d5-420c-9ab9-3eb387788042 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:35.446255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c97c53-51d5-420c-9ab9-3eb387788042 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:35.446554 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c97c53-51d5-420c-9ab9-3eb387788042 None admin] PATH_INFO: `/v3/projects/aee657be89e3474aaf6d4c76a77d4152` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:35.447822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c97c53-51d5-420c-9ab9-3eb387788042 None admin] RBAC: Authorizing `identity:get_project(project_id=aee657be89e3474aaf6d4c76a77d4152)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:35.449272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c97c53-51d5-420c-9ab9-3eb387788042 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:35.451285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 556/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:46:35 2022] GET /identity/v3/projects/aee657be89e3474aaf6d4c76a77d4152 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:35.457683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed7468-e557-451e-aaf3-1986e21e5c72 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:35.549010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:35.575171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed7468-e557-451e-aaf3-1986e21e5c72 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:35.611862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c8512-663b-4b33-8655-682c203b0b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:35.612891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c8512-663b-4b33-8655-682c203b0b5c None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:35.613273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c8512-663b-4b33-8655-682c203b0b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:35.613630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c8512-663b-4b33-8655-682c203b0b5c None admin] PATH_INFO: `/v3/projects/aee657be89e3474aaf6d4c76a77d4152/users/63dc8a977d3c4d10be8e1cef85e85e5b/roles/f819d0f29adf41a49fa045dc3c031527` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:35.616060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c8512-663b-4b33-8655-682c203b0b5c None admin] RBAC: Authorizing `identity:create_grant(project_id=aee657be89e3474aaf6d4c76a77d4152, user_id=63dc8a977d3c4d10be8e1cef85e85e5b, role_id=f819d0f29adf41a49fa045dc3c031527)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:35.617543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c8512-663b-4b33-8655-682c203b0b5c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:35.633606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 567/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:46:35 2022] PUT /identity/v3/projects/aee657be89e3474aaf6d4c76a77d4152/users/63dc8a977d3c4d10be8e1cef85e85e5b/roles/f819d0f29adf41a49fa045dc3c031527 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:46:37.715216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a7b1f-bf4e-4da2-8487-d2aa10c679ce None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:37.715867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a7b1f-bf4e-4da2-8487-d2aa10c679ce None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:37.715867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a7b1f-bf4e-4da2-8487-d2aa10c679ce None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:37.717320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 557/1124] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:37.727284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5ff40-7f2b-4040-9718-5383a908cbf4 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:37.727868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5ff40-7f2b-4040-9718-5383a908cbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:37.728173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5ff40-7f2b-4040-9718-5383a908cbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:37.747578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-42a5ff40-7f2b-4040-9718-5383a908cbf4 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:37.748787 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-42a5ff40-7f2b-4040-9718-5383a908cbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:37.823915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 568/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:37.849340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a7b1f-bf4e-4da2-8487-d2aa10c679ce None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:37.967039 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a7b1f-bf4e-4da2-8487-d2aa10c679ce None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:37.980327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:38.007212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040ebb2-1f30-46d9-b1b8-558ae99827d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:38.008101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040ebb2-1f30-46d9-b1b8-558ae99827d3 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:38.008373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040ebb2-1f30-46d9-b1b8-558ae99827d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:38.008641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040ebb2-1f30-46d9-b1b8-558ae99827d3 None admin] PATH_INFO: `/v3/roles/f819d0f29adf41a49fa045dc3c031527` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:38.010144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040ebb2-1f30-46d9-b1b8-558ae99827d3 None admin] RBAC: Authorizing `identity:get_role(role_id=f819d0f29adf41a49fa045dc3c031527)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:38.011555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040ebb2-1f30-46d9-b1b8-558ae99827d3 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:38.012986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 558/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:37 2022] GET /identity/v3/roles/f819d0f29adf41a49fa045dc3c031527 => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:46:38.019455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5ff40-7f2b-4040-9718-5383a908cbf4 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:38.089878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:38.130551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5ff40-7f2b-4040-9718-5383a908cbf4 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:38.168379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b758e11-1213-44cf-9a9e-127034f2ccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:38.169409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b758e11-1213-44cf-9a9e-127034f2ccca None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:38.169736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b758e11-1213-44cf-9a9e-127034f2ccca None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:38.170050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b758e11-1213-44cf-9a9e-127034f2ccca None admin] PATH_INFO: `/v3/users/63dc8a977d3c4d10be8e1cef85e85e5b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:38.171372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b758e11-1213-44cf-9a9e-127034f2ccca None admin] RBAC: Authorizing `identity:get_user(user_id=63dc8a977d3c4d10be8e1cef85e85e5b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:38.173072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b758e11-1213-44cf-9a9e-127034f2ccca None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:38.174910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 569/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:38 2022] GET /identity/v3/users/63dc8a977d3c4d10be8e1cef85e85e5b => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:38.181213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040ebb2-1f30-46d9-b1b8-558ae99827d3 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:38.290525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040ebb2-1f30-46d9-b1b8-558ae99827d3 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:38.325647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132955af-3eff-4f46-bfe5-ae03f1ad1194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:38.326287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132955af-3eff-4f46-bfe5-ae03f1ad1194 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:38.326586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132955af-3eff-4f46-bfe5-ae03f1ad1194 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:38.326865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132955af-3eff-4f46-bfe5-ae03f1ad1194 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:38.374160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:38.397904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132955af-3eff-4f46-bfe5-ae03f1ad1194 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:38.399403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132955af-3eff-4f46-bfe5-ae03f1ad1194 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:38.502001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 559/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:46:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:46:38.508773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b758e11-1213-44cf-9a9e-127034f2ccca None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:38.681296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b758e11-1213-44cf-9a9e-127034f2ccca None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:38.744288 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75939c1d-5446-4613-a766-37f6e7242072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:38.745739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75939c1d-5446-4613-a766-37f6e7242072 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:38.746218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75939c1d-5446-4613-a766-37f6e7242072 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:38.746734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75939c1d-5446-4613-a766-37f6e7242072 None admin] PATH_INFO: `/v3/projects/aee657be89e3474aaf6d4c76a77d4152` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:38.748841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75939c1d-5446-4613-a766-37f6e7242072 None admin] RBAC: Authorizing `identity:get_project(project_id=aee657be89e3474aaf6d4c76a77d4152)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:38.751650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75939c1d-5446-4613-a766-37f6e7242072 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:38.754947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 570/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:46:38 2022] GET /identity/v3/projects/aee657be89e3474aaf6d4c76a77d4152 => generated 311 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:38.761462 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132955af-3eff-4f46-bfe5-ae03f1ad1194 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:38.850149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:38.874350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132955af-3eff-4f46-bfe5-ae03f1ad1194 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:38.912378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c144e08-0d7f-46a9-a381-fc5d1cc44b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:38.913054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c144e08-0d7f-46a9-a381-fc5d1cc44b4e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:38.913339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c144e08-0d7f-46a9-a381-fc5d1cc44b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:38.913613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c144e08-0d7f-46a9-a381-fc5d1cc44b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:38.914461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c144e08-0d7f-46a9-a381-fc5d1cc44b4e None admin] RBAC: Adding query filter params (role.id=f819d0f29adf41a49fa045dc3c031527, scope.project.id=aee657be89e3474aaf6d4c76a77d4152, user.id=63dc8a977d3c4d10be8e1cef85e85e5b) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:38.914810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c144e08-0d7f-46a9-a381-fc5d1cc44b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:38.916294 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c144e08-0d7f-46a9-a381-fc5d1cc44b4e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:38.923529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 560/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:46:38 2022] GET /identity/v3/role_assignments?user.id=63dc8a977d3c4d10be8e1cef85e85e5b&scope.project.id=aee657be89e3474aaf6d4c76a77d4152&role.id=f819d0f29adf41a49fa045dc3c031527 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:41.002519 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafce052-c78c-4ec4-9fcf-5b625f90c50e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:41.003171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafce052-c78c-4ec4-9fcf-5b625f90c50e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:41.003171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafce052-c78c-4ec4-9fcf-5b625f90c50e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:41.004689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 571/1131] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:41.011048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcfa41-bbd9-429c-8255-50c9085047e4 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:41.011824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcfa41-bbd9-429c-8255-50c9085047e4 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:41.011824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcfa41-bbd9-429c-8255-50c9085047e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:41.036156 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-cfdcfa41-bbd9-429c-8255-50c9085047e4 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:41.037901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cfdcfa41-bbd9-429c-8255-50c9085047e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:41.123482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 561/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:46:41.147867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafce052-c78c-4ec4-9fcf-5b625f90c50e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:41.151047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:41.259421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafce052-c78c-4ec4-9fcf-5b625f90c50e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:41.295340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc4d91-92e1-42d9-9a2f-4b2b0ae92b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:41.296150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc4d91-92e1-42d9-9a2f-4b2b0ae92b62 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:41.296422 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc4d91-92e1-42d9-9a2f-4b2b0ae92b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:41.296720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc4d91-92e1-42d9-9a2f-4b2b0ae92b62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:41.298039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc4d91-92e1-42d9-9a2f-4b2b0ae92b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:41.299537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc4d91-92e1-42d9-9a2f-4b2b0ae92b62 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:41.301199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 572/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:41.307418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcfa41-bbd9-429c-8255-50c9085047e4 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:41.397883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:41.434824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcfa41-bbd9-429c-8255-50c9085047e4 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:41.472324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b56041-a37a-47b8-b661-9de09f3c08fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:41.473373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b56041-a37a-47b8-b661-9de09f3c08fd None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:41.473654 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b56041-a37a-47b8-b661-9de09f3c08fd None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:41.473927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b56041-a37a-47b8-b661-9de09f3c08fd None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:41.474688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b56041-a37a-47b8-b661-9de09f3c08fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:41.476146 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b56041-a37a-47b8-b661-9de09f3c08fd None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:41.504262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 562/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:46:41 2022] POST /identity/v3/projects => generated 311 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:46:43.578081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6801a5-9ee4-48e4-9fed-142ef5584d0c None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:43.578716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6801a5-9ee4-48e4-9fed-142ef5584d0c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:43.578862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6801a5-9ee4-48e4-9fed-142ef5584d0c None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:43.580358 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 573/1135] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:43.586098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9647e12-8880-4ffd-b8f3-d801f598cc96 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:43.586644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9647e12-8880-4ffd-b8f3-d801f598cc96 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:43.586644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9647e12-8880-4ffd-b8f3-d801f598cc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:43.604944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a9647e12-8880-4ffd-b8f3-d801f598cc96 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:43.606153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a9647e12-8880-4ffd-b8f3-d801f598cc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:43.675517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 563/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:43.701575 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6801a5-9ee4-48e4-9fed-142ef5584d0c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:43.775885 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:43.810468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6801a5-9ee4-48e4-9fed-142ef5584d0c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:43.847248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51521000-b5f3-4682-b79c-3496429d30c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:43.848315 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51521000-b5f3-4682-b79c-3496429d30c4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:43.848591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51521000-b5f3-4682-b79c-3496429d30c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:43.848863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51521000-b5f3-4682-b79c-3496429d30c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:43.850105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51521000-b5f3-4682-b79c-3496429d30c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:43.851580 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51521000-b5f3-4682-b79c-3496429d30c4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:43.853219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 574/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:46:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:43.859489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9647e12-8880-4ffd-b8f3-d801f598cc96 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:43.915616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:43.963644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9647e12-8880-4ffd-b8f3-d801f598cc96 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:43.999836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f13c2-9dae-4deb-b76f-489db115d5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:44.000640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f13c2-9dae-4deb-b76f-489db115d5e1 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:44.000989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f13c2-9dae-4deb-b76f-489db115d5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:44.001270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f13c2-9dae-4deb-b76f-489db115d5e1 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:44.002001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f13c2-9dae-4deb-b76f-489db115d5e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:44.003481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f13c2-9dae-4deb-b76f-489db115d5e1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:44.025115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 564/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:46:43 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:46:46.122869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebd2fa-e3f0-41e4-a421-4c038241b0c4 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:46.123689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebd2fa-e3f0-41e4-a421-4c038241b0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:46.123842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebd2fa-e3f0-41e4-a421-4c038241b0c4 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:46.125581 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 575/1139] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:46.131288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefa029-ad47-4fca-99ef-fe61b4cc2cd4 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:46.131797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefa029-ad47-4fca-99ef-fe61b4cc2cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:46.131797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefa029-ad47-4fca-99ef-fe61b4cc2cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:46.150452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-daefa029-ad47-4fca-99ef-fe61b4cc2cd4 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:46.151758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-daefa029-ad47-4fca-99ef-fe61b4cc2cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:46.228734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 565/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:46.253407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebd2fa-e3f0-41e4-a421-4c038241b0c4 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:46.368097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebd2fa-e3f0-41e4-a421-4c038241b0c4 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:46.405977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b865576-3dc6-4789-acd9-daca1cf8d205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:46.407010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b865576-3dc6-4789-acd9-daca1cf8d205 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:46.407260 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b865576-3dc6-4789-acd9-daca1cf8d205 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:46.407553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b865576-3dc6-4789-acd9-daca1cf8d205 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:46.415407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:46.419814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b865576-3dc6-4789-acd9-daca1cf8d205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-1b865576-3dc6-4789-acd9-daca1cf8d205 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:46:46.427895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:46.429052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:46.430203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 576/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:46:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:46.436113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefa029-ad47-4fca-99ef-fe61b4cc2cd4 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:46.480460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:46.542804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefa029-ad47-4fca-99ef-fe61b4cc2cd4 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:46.579818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f596c-7273-4f25-a77f-d22606c53610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:46.580521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f596c-7273-4f25-a77f-d22606c53610 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:46.580799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f596c-7273-4f25-a77f-d22606c53610 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:46.581076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f596c-7273-4f25-a77f-d22606c53610 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:46.581824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f596c-7273-4f25-a77f-d22606c53610 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:46.582111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f596c-7273-4f25-a77f-d22606c53610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:46.583530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f596c-7273-4f25-a77f-d22606c53610 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:46.592415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 566/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:46:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:46.599021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b865576-3dc6-4789-acd9-daca1cf8d205 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:46.701774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b865576-3dc6-4789-acd9-daca1cf8d205 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:46.740890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fecc04-4d62-4d19-8976-2550498e5bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:46.741775 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fecc04-4d62-4d19-8976-2550498e5bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:46.742051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fecc04-4d62-4d19-8976-2550498e5bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:46.742319 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fecc04-4d62-4d19-8976-2550498e5bb6 None admin] PATH_INFO: `/v3/users/748176ad9e75401380bd2a8d939af1b1` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:46.756845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fecc04-4d62-4d19-8976-2550498e5bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=748176ad9e75401380bd2a8d939af1b1)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:46.758321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fecc04-4d62-4d19-8976-2550498e5bb6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:46.760106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 577/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:46 2022] GET /identity/v3/users/748176ad9e75401380bd2a8d939af1b1 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:46.766177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f596c-7273-4f25-a77f-d22606c53610 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:46.876364 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f596c-7273-4f25-a77f-d22606c53610 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:46.878085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:46.914785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8030cb3-469e-422e-9b31-31acf5b819ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:46.915441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8030cb3-469e-422e-9b31-31acf5b819ce None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:46.915718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8030cb3-469e-422e-9b31-31acf5b819ce None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:46.915986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8030cb3-469e-422e-9b31-31acf5b819ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:46.992511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8030cb3-469e-422e-9b31-31acf5b819ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:46.993870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8030cb3-469e-422e-9b31-31acf5b819ce None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:47.101448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 567/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:46:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:46:47.107893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fecc04-4d62-4d19-8976-2550498e5bb6 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:47.173526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:47.220043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fecc04-4d62-4d19-8976-2550498e5bb6 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:47.256656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c7d9f-ef24-4824-bed9-408856d045e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:47.257412 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c7d9f-ef24-4824-bed9-408856d045e5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:47.257685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c7d9f-ef24-4824-bed9-408856d045e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:47.257951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c7d9f-ef24-4824-bed9-408856d045e5 None admin] PATH_INFO: `/v3/projects/c95fe67df5ff4b0c98470c728ffa7d50` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:47.259195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c7d9f-ef24-4824-bed9-408856d045e5 None admin] RBAC: Authorizing `identity:get_project(project_id=c95fe67df5ff4b0c98470c728ffa7d50)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:47.260632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c7d9f-ef24-4824-bed9-408856d045e5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:47.262339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 578/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:46:47 2022] GET /identity/v3/projects/c95fe67df5ff4b0c98470c728ffa7d50 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:47.269414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8030cb3-469e-422e-9b31-31acf5b819ce None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:47.271786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:47.374482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8030cb3-469e-422e-9b31-31acf5b819ce None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:47.411847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ead74a-ad11-45ae-a5cc-fcf626624dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:47.412511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead74a-ad11-45ae-a5cc-fcf626624dcb None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:47.412787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead74a-ad11-45ae-a5cc-fcf626624dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:47.413057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead74a-ad11-45ae-a5cc-fcf626624dcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:47.413884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ead74a-ad11-45ae-a5cc-fcf626624dcb None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=c95fe67df5ff4b0c98470c728ffa7d50, user.id=748176ad9e75401380bd2a8d939af1b1) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:47.414189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ead74a-ad11-45ae-a5cc-fcf626624dcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:47.415665 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ead74a-ad11-45ae-a5cc-fcf626624dcb None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:47.421693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 568/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:46:47 2022] GET /identity/v3/role_assignments?user.id=748176ad9e75401380bd2a8d939af1b1&scope.project.id=c95fe67df5ff4b0c98470c728ffa7d50&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:49.495090 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5bde5-3fd8-42b0-8b55-d66331c3808b None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:49.495672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5bde5-3fd8-42b0-8b55-d66331c3808b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:49.495672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5bde5-3fd8-42b0-8b55-d66331c3808b None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:49.497072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 579/1147] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:49.503886 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a7d86-6407-4c18-a5c9-9da7f39bd625 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:49.504356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a7d86-6407-4c18-a5c9-9da7f39bd625 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:49.504467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a7d86-6407-4c18-a5c9-9da7f39bd625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:49.522855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-968a7d86-6407-4c18-a5c9-9da7f39bd625 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:49.524074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-968a7d86-6407-4c18-a5c9-9da7f39bd625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:49.601252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:49.610111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 569/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:49.634227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5bde5-3fd8-42b0-8b55-d66331c3808b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:49.756340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5bde5-3fd8-42b0-8b55-d66331c3808b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:49.788446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:49.798212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21365d8-d60b-488f-ad65-a97f7a2a0827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:49.799056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21365d8-d60b-488f-ad65-a97f7a2a0827 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:49.799332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21365d8-d60b-488f-ad65-a97f7a2a0827 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:49.799602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21365d8-d60b-488f-ad65-a97f7a2a0827 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:49.810361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21365d8-d60b-488f-ad65-a97f7a2a0827 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f21365d8-d60b-488f-ad65-a97f7a2a0827 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:46:49.818373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:49.820091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:49.820990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 580/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:46:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:49.826374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a7d86-6407-4c18-a5c9-9da7f39bd625 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:49.947922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a7d86-6407-4c18-a5c9-9da7f39bd625 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:49.987878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904a242-60f9-4a26-a5af-e44889c083c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:49.988596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904a242-60f9-4a26-a5af-e44889c083c2 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:49.988923 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904a242-60f9-4a26-a5af-e44889c083c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:49.989218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904a242-60f9-4a26-a5af-e44889c083c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:49.989967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904a242-60f9-4a26-a5af-e44889c083c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:49.990297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904a242-60f9-4a26-a5af-e44889c083c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:49.991787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904a242-60f9-4a26-a5af-e44889c083c2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:50.001025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 570/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:46:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:50.007908 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21365d8-d60b-488f-ad65-a97f7a2a0827 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:50.120532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21365d8-d60b-488f-ad65-a97f7a2a0827 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:50.160686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16668b2e-1254-40f5-9493-5cf64e9e59d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:50.161570 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16668b2e-1254-40f5-9493-5cf64e9e59d5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:50.161849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16668b2e-1254-40f5-9493-5cf64e9e59d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:50.162121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16668b2e-1254-40f5-9493-5cf64e9e59d5 None admin] PATH_INFO: `/v3/users/748176ad9e75401380bd2a8d939af1b1` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:50.163359 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16668b2e-1254-40f5-9493-5cf64e9e59d5 None admin] RBAC: Authorizing `identity:get_user(user_id=748176ad9e75401380bd2a8d939af1b1)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:50.164807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16668b2e-1254-40f5-9493-5cf64e9e59d5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:50.166521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 581/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:50 2022] GET /identity/v3/users/748176ad9e75401380bd2a8d939af1b1 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:50.173986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904a242-60f9-4a26-a5af-e44889c083c2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:50.247501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:50.293663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904a242-60f9-4a26-a5af-e44889c083c2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:50.338717 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669aa6b-dc29-405b-a56c-11ebc1bda145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:50.339606 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669aa6b-dc29-405b-a56c-11ebc1bda145 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:50.340011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669aa6b-dc29-405b-a56c-11ebc1bda145 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:50.340291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669aa6b-dc29-405b-a56c-11ebc1bda145 None admin] PATH_INFO: `/v3/projects/c95fe67df5ff4b0c98470c728ffa7d50` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:50.341593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669aa6b-dc29-405b-a56c-11ebc1bda145 None admin] RBAC: Authorizing `identity:get_project(project_id=c95fe67df5ff4b0c98470c728ffa7d50)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:50.343105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669aa6b-dc29-405b-a56c-11ebc1bda145 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:50.344846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 571/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:46:50 2022] GET /identity/v3/projects/c95fe67df5ff4b0c98470c728ffa7d50 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:50.351695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16668b2e-1254-40f5-9493-5cf64e9e59d5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:50.407424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:50.461761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16668b2e-1254-40f5-9493-5cf64e9e59d5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:50.501551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498957be-2133-4798-86b2-8bb033274d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:50.502447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498957be-2133-4798-86b2-8bb033274d29 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:50.502754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498957be-2133-4798-86b2-8bb033274d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:50.503022 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498957be-2133-4798-86b2-8bb033274d29 None admin] PATH_INFO: `/v3/projects/c95fe67df5ff4b0c98470c728ffa7d50/users/748176ad9e75401380bd2a8d939af1b1/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:50.505012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498957be-2133-4798-86b2-8bb033274d29 None admin] RBAC: Authorizing `identity:create_grant(project_id=c95fe67df5ff4b0c98470c728ffa7d50, user_id=748176ad9e75401380bd2a8d939af1b1, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:50.506591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498957be-2133-4798-86b2-8bb033274d29 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:50.523646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 582/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:46:50 2022] PUT /identity/v3/projects/c95fe67df5ff4b0c98470c728ffa7d50/users/748176ad9e75401380bd2a8d939af1b1/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:46:52.682612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eaf64-54e7-4243-a62c-75be3822f1da None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:52.683185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eaf64-54e7-4243-a62c-75be3822f1da None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:52.683300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eaf64-54e7-4243-a62c-75be3822f1da None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:52.684716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 572/1154] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:52.690940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c7d1f-1e53-4cf6-b20e-fbcc42286157 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:52.691527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c7d1f-1e53-4cf6-b20e-fbcc42286157 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:52.691527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c7d1f-1e53-4cf6-b20e-fbcc42286157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:52.712597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a41c7d1f-1e53-4cf6-b20e-fbcc42286157 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:52.713823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a41c7d1f-1e53-4cf6-b20e-fbcc42286157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:52.792877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 583/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:52.818523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426eaf64-54e7-4243-a62c-75be3822f1da None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:52.935627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426eaf64-54e7-4243-a62c-75be3822f1da None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:52.963856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:52.974472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c1c90-cbed-4d5f-a784-17d14aade3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:52.976603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1c90-cbed-4d5f-a784-17d14aade3ae None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:52.976908 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1c90-cbed-4d5f-a784-17d14aade3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:52.977164 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c1c90-cbed-4d5f-a784-17d14aade3ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:52.989050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c1c90-cbed-4d5f-a784-17d14aade3ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-016c1c90-cbed-4d5f-a784-17d14aade3ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:46:52.996660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:52.998411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:46:52.999907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 573/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:46:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:53.004604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c7d1f-1e53-4cf6-b20e-fbcc42286157 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:53.060229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:53.125523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c7d1f-1e53-4cf6-b20e-fbcc42286157 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:53.164973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9d439-353b-461e-9bf5-ee13e0f580bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:53.165659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9d439-353b-461e-9bf5-ee13e0f580bd None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:53.165940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9d439-353b-461e-9bf5-ee13e0f580bd None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:53.166211 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9d439-353b-461e-9bf5-ee13e0f580bd None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:53.167006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9d439-353b-461e-9bf5-ee13e0f580bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:53.167313 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9d439-353b-461e-9bf5-ee13e0f580bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:53.168711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9d439-353b-461e-9bf5-ee13e0f580bd None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:53.178023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 584/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:46:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:53.184350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c1c90-cbed-4d5f-a784-17d14aade3ae None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:53.291370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c1c90-cbed-4d5f-a784-17d14aade3ae None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:53.328131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29a65d-8594-4f5f-9a23-71f86ce98373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:53.329073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29a65d-8594-4f5f-9a23-71f86ce98373 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:53.329333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29a65d-8594-4f5f-9a23-71f86ce98373 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:53.329626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29a65d-8594-4f5f-9a23-71f86ce98373 None admin] PATH_INFO: `/v3/users/748176ad9e75401380bd2a8d939af1b1` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:53.330917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29a65d-8594-4f5f-9a23-71f86ce98373 None admin] RBAC: Authorizing `identity:get_user(user_id=748176ad9e75401380bd2a8d939af1b1)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:53.332397 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29a65d-8594-4f5f-9a23-71f86ce98373 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:53.334165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 574/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:46:53 2022] GET /identity/v3/users/748176ad9e75401380bd2a8d939af1b1 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:53.341120 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9d439-353b-461e-9bf5-ee13e0f580bd None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:53.471676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9d439-353b-461e-9bf5-ee13e0f580bd None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:53.474688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:53.508567 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389503a5-2730-470d-86b1-45c132f44045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:53.509218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389503a5-2730-470d-86b1-45c132f44045 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:53.509499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389503a5-2730-470d-86b1-45c132f44045 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:53.509770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389503a5-2730-470d-86b1-45c132f44045 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:53.579008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389503a5-2730-470d-86b1-45c132f44045 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:53.580399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389503a5-2730-470d-86b1-45c132f44045 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:53.684355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 585/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:46:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:46:53.691707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29a65d-8594-4f5f-9a23-71f86ce98373 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:53.752867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:53.802693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29a65d-8594-4f5f-9a23-71f86ce98373 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:53.839666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fecdf5-a5a2-42d1-9594-d52ad2e54e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:53.840430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fecdf5-a5a2-42d1-9594-d52ad2e54e67 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:53.840698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fecdf5-a5a2-42d1-9594-d52ad2e54e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:53.840974 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fecdf5-a5a2-42d1-9594-d52ad2e54e67 None admin] PATH_INFO: `/v3/projects/c95fe67df5ff4b0c98470c728ffa7d50` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:53.842196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fecdf5-a5a2-42d1-9594-d52ad2e54e67 None admin] RBAC: Authorizing `identity:get_project(project_id=c95fe67df5ff4b0c98470c728ffa7d50)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:53.844368 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fecdf5-a5a2-42d1-9594-d52ad2e54e67 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:53.846115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 575/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:46:53 2022] GET /identity/v3/projects/c95fe67df5ff4b0c98470c728ffa7d50 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:53.852846 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389503a5-2730-470d-86b1-45c132f44045 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:53.856089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:53.959061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389503a5-2730-470d-86b1-45c132f44045 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:53.994518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee95de-110e-4ceb-9aea-8dd1765b5f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:53.995197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee95de-110e-4ceb-9aea-8dd1765b5f63 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:53.995475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee95de-110e-4ceb-9aea-8dd1765b5f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:53.995746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee95de-110e-4ceb-9aea-8dd1765b5f63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:53.996577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee95de-110e-4ceb-9aea-8dd1765b5f63 None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=c95fe67df5ff4b0c98470c728ffa7d50, user.id=748176ad9e75401380bd2a8d939af1b1) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:53.996884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee95de-110e-4ceb-9aea-8dd1765b5f63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:53.998343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee95de-110e-4ceb-9aea-8dd1765b5f63 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:54.004761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 586/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:46:53 2022] GET /identity/v3/role_assignments?user.id=748176ad9e75401380bd2a8d939af1b1&scope.project.id=c95fe67df5ff4b0c98470c728ffa7d50&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:56.133384 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9754a-bd8b-43a1-8e12-b28c4d746a18 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:56.133944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9754a-bd8b-43a1-8e12-b28c4d746a18 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:56.133944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9754a-bd8b-43a1-8e12-b28c4d746a18 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:56.135349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 576/1162] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:56.140862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adab491-4d76-4ff0-ad2c-b6857a1e9054 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:56.141330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adab491-4d76-4ff0-ad2c-b6857a1e9054 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:56.141440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adab491-4d76-4ff0-ad2c-b6857a1e9054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:56.160125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8adab491-4d76-4ff0-ad2c-b6857a1e9054 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:56.161358 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8adab491-4d76-4ff0-ad2c-b6857a1e9054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:56.237363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:56.244146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 587/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:56.268897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9754a-bd8b-43a1-8e12-b28c4d746a18 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:56.384316 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9754a-bd8b-43a1-8e12-b28c4d746a18 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:56.412479 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:56.425893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faecd6b-8b43-4e92-8f7a-8920caf96212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:56.426607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecd6b-8b43-4e92-8f7a-8920caf96212 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:56.426882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecd6b-8b43-4e92-8f7a-8920caf96212 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:56.427146 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecd6b-8b43-4e92-8f7a-8920caf96212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:56.502827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faecd6b-8b43-4e92-8f7a-8920caf96212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:56.504282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faecd6b-8b43-4e92-8f7a-8920caf96212 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:56.616342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 577/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:46:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:46:56.623193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adab491-4d76-4ff0-ad2c-b6857a1e9054 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:56.734985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adab491-4d76-4ff0-ad2c-b6857a1e9054 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:56.771271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143502a-1a83-4867-86ce-5f578842f370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:56.772093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143502a-1a83-4867-86ce-5f578842f370 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:56.772372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143502a-1a83-4867-86ce-5f578842f370 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:56.772642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143502a-1a83-4867-86ce-5f578842f370 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:56.780121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143502a-1a83-4867-86ce-5f578842f370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:56.781946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143502a-1a83-4867-86ce-5f578842f370 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-c143502a-1a83-4867-86ce-5f578842f370 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:46:56.796592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:46:56.797992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:46:56.799812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 588/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 09:46:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:46:56.804364 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faecd6b-8b43-4e92-8f7a-8920caf96212 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:56.831375 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:56.912516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faecd6b-8b43-4e92-8f7a-8920caf96212 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:56.948412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbae65e-e22a-45f7-8ae5-5f09500ac267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:56.949083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae65e-e22a-45f7-8ae5-5f09500ac267 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:56.949359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae65e-e22a-45f7-8ae5-5f09500ac267 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:56.949632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbae65e-e22a-45f7-8ae5-5f09500ac267 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:56.950360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbae65e-e22a-45f7-8ae5-5f09500ac267 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:46:56.950691 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbae65e-e22a-45f7-8ae5-5f09500ac267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:56.952068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbae65e-e22a-45f7-8ae5-5f09500ac267 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:56.960897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 578/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 09:46:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:46:59.010891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d0ed8-6bd0-4ded-9ed3-3b0feeffbf9a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:59.011489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d0ed8-6bd0-4ded-9ed3-3b0feeffbf9a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:59.011489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d0ed8-6bd0-4ded-9ed3-3b0feeffbf9a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:59.012788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 589/1167] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:46:59.019512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4ff76-565f-4e30-a023-1462fffbe29c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:59.019983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4ff76-565f-4e30-a023-1462fffbe29c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:59.020075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4ff76-565f-4e30-a023-1462fffbe29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:59.037403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6bb4ff76-565f-4e30-a023-1462fffbe29c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:46:59.038611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6bb4ff76-565f-4e30-a023-1462fffbe29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:46:59.113024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 579/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:46:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:46:59.136687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d0ed8-6bd0-4ded-9ed3-3b0feeffbf9a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:46:59.198105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:46:59.243586 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d0ed8-6bd0-4ded-9ed3-3b0feeffbf9a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:46:59.281779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89faab33-fc44-4024-a12b-0de6a7b0114b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:46:59.282362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89faab33-fc44-4024-a12b-0de6a7b0114b None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:46:59.282666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89faab33-fc44-4024-a12b-0de6a7b0114b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:46:59.282944 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89faab33-fc44-4024-a12b-0de6a7b0114b None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:46:59.283731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89faab33-fc44-4024-a12b-0de6a7b0114b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:46:59.285155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89faab33-fc44-4024-a12b-0de6a7b0114b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:46:59.302538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 590/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Wed Jun 29 09:46:59 2022] POST /identity/v3/domains => generated 267 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:47:01.404635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6ee2-9e4b-414b-866d-2c36d976d55b None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:01.405183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6ee2-9e4b-414b-866d-2c36d976d55b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:01.405183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6ee2-9e4b-414b-866d-2c36d976d55b None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:01.406789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 580/1170] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:47:01.412114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17b0a8-eeb3-482c-8cd8-b2187b2d9a42 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:01.412574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17b0a8-eeb3-482c-8cd8-b2187b2d9a42 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:01.412672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17b0a8-eeb3-482c-8cd8-b2187b2d9a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:01.430818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ac17b0a8-eeb3-482c-8cd8-b2187b2d9a42 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:47:01.432022 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ac17b0a8-eeb3-482c-8cd8-b2187b2d9a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:47:01.503119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 591/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:47:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:47:01.527026 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d6ee2-9e4b-414b-866d-2c36d976d55b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:01.548047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:01.641265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d6ee2-9e4b-414b-866d-2c36d976d55b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:01.679491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c749640-a23d-40c3-bdd7-e86521859474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:01.680274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749640-a23d-40c3-bdd7-e86521859474 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:01.680553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749640-a23d-40c3-bdd7-e86521859474 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:01.680825 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749640-a23d-40c3-bdd7-e86521859474 None admin] PATH_INFO: `/v3/domains/1ee1267c730346948097b71cae25f6b0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:01.692483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c749640-a23d-40c3-bdd7-e86521859474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1ee1267c730346948097b71cae25f6b0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:01.693969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c749640-a23d-40c3-bdd7-e86521859474 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:01.695792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 581/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Jun 29 09:47:01 2022] GET /identity/v3/domains/1ee1267c730346948097b71cae25f6b0 => generated 267 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:01.702033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17b0a8-eeb3-482c-8cd8-b2187b2d9a42 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:01.759668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:01.837513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17b0a8-eeb3-482c-8cd8-b2187b2d9a42 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:01.876955 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e07f3d-2f5f-4570-9dfe-af924b03d3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:01.877628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e07f3d-2f5f-4570-9dfe-af924b03d3b1 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:01.877907 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e07f3d-2f5f-4570-9dfe-af924b03d3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:01.878174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e07f3d-2f5f-4570-9dfe-af924b03d3b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:01.878932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e07f3d-2f5f-4570-9dfe-af924b03d3b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:01.880382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e07f3d-2f5f-4570-9dfe-af924b03d3b1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:01.909129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 592/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:47:01 2022] POST /identity/v3/projects => generated 361 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:47:03.983512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571062da-0271-48fc-b7a1-c0e64fb5599a None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:03.984108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571062da-0271-48fc-b7a1-c0e64fb5599a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:03.984243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571062da-0271-48fc-b7a1-c0e64fb5599a None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:03.985669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 582/1174] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:47:03.991858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870821e-dff1-460d-b71d-b33bd8adb1f4 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:03.992282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870821e-dff1-460d-b71d-b33bd8adb1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:03.992407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870821e-dff1-460d-b71d-b33bd8adb1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:04.010845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6870821e-dff1-460d-b71d-b33bd8adb1f4 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:47:04.012028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6870821e-dff1-460d-b71d-b33bd8adb1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:47:04.081936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 593/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:47:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:47:04.108359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571062da-0271-48fc-b7a1-c0e64fb5599a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:04.197234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:04.222139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571062da-0271-48fc-b7a1-c0e64fb5599a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:04.258913 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8067ab5-48d0-433c-b1b1-9918f8037237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:04.259699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8067ab5-48d0-433c-b1b1-9918f8037237 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:04.259980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8067ab5-48d0-433c-b1b1-9918f8037237 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:04.260255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8067ab5-48d0-433c-b1b1-9918f8037237 None admin] PATH_INFO: `/v3/domains/1ee1267c730346948097b71cae25f6b0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:04.261513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8067ab5-48d0-433c-b1b1-9918f8037237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1ee1267c730346948097b71cae25f6b0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:04.263000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8067ab5-48d0-433c-b1b1-9918f8037237 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:04.264667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 583/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Jun 29 09:47:04 2022] GET /identity/v3/domains/1ee1267c730346948097b71cae25f6b0 => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:04.275478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870821e-dff1-460d-b71d-b33bd8adb1f4 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:04.328726 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:04.408208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870821e-dff1-460d-b71d-b33bd8adb1f4 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:04.443393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eab75-b5aa-49dc-a848-9ee53908fa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:04.444071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eab75-b5aa-49dc-a848-9ee53908fa2a None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:04.444349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eab75-b5aa-49dc-a848-9ee53908fa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:04.444633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eab75-b5aa-49dc-a848-9ee53908fa2a None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:04.445348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61eab75-b5aa-49dc-a848-9ee53908fa2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:04.446833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61eab75-b5aa-49dc-a848-9ee53908fa2a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:04.468243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 594/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:47:04 2022] POST /identity/v3/users => generated 311 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:47:06.556121 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b4d5a-1566-4e3e-81ef-b1b367ea1f02 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:06.556671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b4d5a-1566-4e3e-81ef-b1b367ea1f02 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:06.557100 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b4d5a-1566-4e3e-81ef-b1b367ea1f02 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:06.559721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 584/1178] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:47:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:47:06.565755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6077c12-6eb7-4517-a48b-9a5ea2c31d4c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:06.566198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6077c12-6eb7-4517-a48b-9a5ea2c31d4c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:06.566288 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6077c12-6eb7-4517-a48b-9a5ea2c31d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:06.584575 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e6077c12-6eb7-4517-a48b-9a5ea2c31d4c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:47:06.585781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e6077c12-6eb7-4517-a48b-9a5ea2c31d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:47:06.658220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 595/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:47:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:47:06.682845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b4d5a-1566-4e3e-81ef-b1b367ea1f02 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:06.795521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b4d5a-1566-4e3e-81ef-b1b367ea1f02 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:06.830979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d386a-98a3-42b7-9ac3-249d296abff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:06.831701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d386a-98a3-42b7-9ac3-249d296abff1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:06.831984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d386a-98a3-42b7-9ac3-249d296abff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:06.832257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d386a-98a3-42b7-9ac3-249d296abff1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:06.842998 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d386a-98a3-42b7-9ac3-249d296abff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-405d386a-98a3-42b7-9ac3-249d296abff1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:47:06.850989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:47:06.852692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:47:06.854325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 585/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:47:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:47:06.858597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6077c12-6eb7-4517-a48b-9a5ea2c31d4c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:06.877502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:06.966501 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6077c12-6eb7-4517-a48b-9a5ea2c31d4c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:07.008199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb30ab-1bf6-458d-ab15-cfa083082f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:07.008914 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb30ab-1bf6-458d-ab15-cfa083082f46 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:07.009193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb30ab-1bf6-458d-ab15-cfa083082f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:07.009471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb30ab-1bf6-458d-ab15-cfa083082f46 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:07.010231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb30ab-1bf6-458d-ab15-cfa083082f46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:47:07.010539 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb30ab-1bf6-458d-ab15-cfa083082f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:07.012006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb30ab-1bf6-458d-ab15-cfa083082f46 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:07.022198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 596/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:47:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:07.028530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d386a-98a3-42b7-9ac3-249d296abff1 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:07.033147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:07.154030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d386a-98a3-42b7-9ac3-249d296abff1 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:07.189183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdfb667-3de1-458a-8916-d85534c6eb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:07.190062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfb667-3de1-458a-8916-d85534c6eb30 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:07.190322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfb667-3de1-458a-8916-d85534c6eb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:07.190657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfb667-3de1-458a-8916-d85534c6eb30 None admin] PATH_INFO: `/v3/users/37e082fb79ff4caf903d4c58ab9b09fe` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:07.206930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdfb667-3de1-458a-8916-d85534c6eb30 None admin] RBAC: Authorizing `identity:get_user(user_id=37e082fb79ff4caf903d4c58ab9b09fe)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:07.208674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdfb667-3de1-458a-8916-d85534c6eb30 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:07.210812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 586/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:47:07 2022] GET /identity/v3/users/37e082fb79ff4caf903d4c58ab9b09fe => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:07.217085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb30ab-1bf6-458d-ab15-cfa083082f46 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:07.302128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:07.325546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb30ab-1bf6-458d-ab15-cfa083082f46 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:07.360717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742e907-c31e-423e-b8f7-12d085752ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:07.361383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742e907-c31e-423e-b8f7-12d085752ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:07.361667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742e907-c31e-423e-b8f7-12d085752ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:07.361944 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742e907-c31e-423e-b8f7-12d085752ad9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:07.433217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742e907-c31e-423e-b8f7-12d085752ad9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:07.434751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742e907-c31e-423e-b8f7-12d085752ad9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:07.522977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:07.546922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 597/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:47:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:47:07.553156 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdfb667-3de1-458a-8916-d85534c6eb30 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:07.616981 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:07.658159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdfb667-3de1-458a-8916-d85534c6eb30 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:07.693743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7560781-73ac-4968-a8bd-db2e89649d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:07.694530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7560781-73ac-4968-a8bd-db2e89649d95 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:07.694907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7560781-73ac-4968-a8bd-db2e89649d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:07.695189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7560781-73ac-4968-a8bd-db2e89649d95 None admin] PATH_INFO: `/v3/projects/c17761004e3b4de3bba61f4163288439` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:07.696426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7560781-73ac-4968-a8bd-db2e89649d95 None admin] RBAC: Authorizing `identity:get_project(project_id=c17761004e3b4de3bba61f4163288439)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:07.697880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7560781-73ac-4968-a8bd-db2e89649d95 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:07.699672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 587/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:47:07 2022] GET /identity/v3/projects/c17761004e3b4de3bba61f4163288439 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:07.706168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742e907-c31e-423e-b8f7-12d085752ad9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:07.809031 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742e907-c31e-423e-b8f7-12d085752ad9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:07.847125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e055bd-9b5e-4e18-8c2a-91db7ebc8bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:07.848183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e055bd-9b5e-4e18-8c2a-91db7ebc8bbe None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:07.848467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e055bd-9b5e-4e18-8c2a-91db7ebc8bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:07.848739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e055bd-9b5e-4e18-8c2a-91db7ebc8bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:07.849577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e055bd-9b5e-4e18-8c2a-91db7ebc8bbe None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=c17761004e3b4de3bba61f4163288439, user.id=37e082fb79ff4caf903d4c58ab9b09fe) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:47:07.849888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e055bd-9b5e-4e18-8c2a-91db7ebc8bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:07.851369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e055bd-9b5e-4e18-8c2a-91db7ebc8bbe None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:07.857359 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 598/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:47:07 2022] GET /identity/v3/role_assignments?user.id=37e082fb79ff4caf903d4c58ab9b09fe&scope.project.id=c17761004e3b4de3bba61f4163288439&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:09.876213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314a75e-0c01-44c3-8bdb-9c84eeabb2f2 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:09.876804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314a75e-0c01-44c3-8bdb-9c84eeabb2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:09.876804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314a75e-0c01-44c3-8bdb-9c84eeabb2f2 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:09.878288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 588/1186] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:47:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:47:09.884395 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a894518-82fa-4201-a2fc-955ef9fbbaf7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:09.884963 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a894518-82fa-4201-a2fc-955ef9fbbaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:09.884963 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a894518-82fa-4201-a2fc-955ef9fbbaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:09.904022 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4a894518-82fa-4201-a2fc-955ef9fbbaf7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:47:09.905343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4a894518-82fa-4201-a2fc-955ef9fbbaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:47:09.946389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:09.982700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 599/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:47:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:47:10.007973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314a75e-0c01-44c3-8bdb-9c84eeabb2f2 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:10.123629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314a75e-0c01-44c3-8bdb-9c84eeabb2f2 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:10.155055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:10.162246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90c7ba-ec02-4957-85ed-941e53de057d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:10.163085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90c7ba-ec02-4957-85ed-941e53de057d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:10.163363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90c7ba-ec02-4957-85ed-941e53de057d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:10.163635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90c7ba-ec02-4957-85ed-941e53de057d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:10.174844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90c7ba-ec02-4957-85ed-941e53de057d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-ce90c7ba-ec02-4957-85ed-941e53de057d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:47:10.182367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:47:10.184089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:47:10.185593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 589/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:47:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:47:10.194544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a894518-82fa-4201-a2fc-955ef9fbbaf7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:10.362951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a894518-82fa-4201-a2fc-955ef9fbbaf7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:10.402697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faac9e-dd79-4882-8950-a67a83710b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:10.403369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faac9e-dd79-4882-8950-a67a83710b12 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:10.403648 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faac9e-dd79-4882-8950-a67a83710b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:10.403922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faac9e-dd79-4882-8950-a67a83710b12 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:10.404676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faac9e-dd79-4882-8950-a67a83710b12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:47:10.404985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faac9e-dd79-4882-8950-a67a83710b12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:10.406879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faac9e-dd79-4882-8950-a67a83710b12 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:10.416161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 600/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:47:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:10.423391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90c7ba-ec02-4957-85ed-941e53de057d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:10.537659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90c7ba-ec02-4957-85ed-941e53de057d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:10.575602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117fecc-07e0-48a4-9fcb-afb469296e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:10.576488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fecc-07e0-48a4-9fcb-afb469296e3b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:10.576771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fecc-07e0-48a4-9fcb-afb469296e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:10.577051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fecc-07e0-48a4-9fcb-afb469296e3b None admin] PATH_INFO: `/v3/users/37e082fb79ff4caf903d4c58ab9b09fe` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:10.578382 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117fecc-07e0-48a4-9fcb-afb469296e3b None admin] RBAC: Authorizing `identity:get_user(user_id=37e082fb79ff4caf903d4c58ab9b09fe)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:10.579869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117fecc-07e0-48a4-9fcb-afb469296e3b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:10.581535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 590/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:47:10 2022] GET /identity/v3/users/37e082fb79ff4caf903d4c58ab9b09fe => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:10.587599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faac9e-dd79-4882-8950-a67a83710b12 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:10.626805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:10.697434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faac9e-dd79-4882-8950-a67a83710b12 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:10.743209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2abd8-4de9-4a2a-af17-796fb0534ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:10.744141 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2abd8-4de9-4a2a-af17-796fb0534ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:10.744443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2abd8-4de9-4a2a-af17-796fb0534ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:10.744718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2abd8-4de9-4a2a-af17-796fb0534ec0 None admin] PATH_INFO: `/v3/projects/c17761004e3b4de3bba61f4163288439` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:10.746315 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2abd8-4de9-4a2a-af17-796fb0534ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=c17761004e3b4de3bba61f4163288439)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:10.747964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2abd8-4de9-4a2a-af17-796fb0534ec0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:10.750666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 601/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:47:10 2022] GET /identity/v3/projects/c17761004e3b4de3bba61f4163288439 => generated 361 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:10.757508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117fecc-07e0-48a4-9fcb-afb469296e3b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:10.828793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:10.875005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117fecc-07e0-48a4-9fcb-afb469296e3b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:10.914459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cfb29-aeba-404c-a9e6-47f953854849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:10.915392 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cfb29-aeba-404c-a9e6-47f953854849 None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:10.915669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cfb29-aeba-404c-a9e6-47f953854849 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:10.915936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cfb29-aeba-404c-a9e6-47f953854849 None admin] PATH_INFO: `/v3/projects/c17761004e3b4de3bba61f4163288439/users/37e082fb79ff4caf903d4c58ab9b09fe/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:10.917940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cfb29-aeba-404c-a9e6-47f953854849 None admin] RBAC: Authorizing `identity:create_grant(project_id=c17761004e3b4de3bba61f4163288439, user_id=37e082fb79ff4caf903d4c58ab9b09fe, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:10.919428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cfb29-aeba-404c-a9e6-47f953854849 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:10.934949 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 591/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:47:10 2022] PUT /identity/v3/projects/c17761004e3b4de3bba61f4163288439/users/37e082fb79ff4caf903d4c58ab9b09fe/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:47:12.995899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222702d-eb32-48d7-83fb-dede417d9710 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:12.996513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222702d-eb32-48d7-83fb-dede417d9710 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:12.996513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222702d-eb32-48d7-83fb-dede417d9710 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:12.997882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 602/1193] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:47:13.004498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dd1e5-72a0-4a1c-abde-c4f0ed2012e0 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:13.005036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dd1e5-72a0-4a1c-abde-c4f0ed2012e0 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:13.005036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dd1e5-72a0-4a1c-abde-c4f0ed2012e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:13.024852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-3b7dd1e5-72a0-4a1c-abde-c4f0ed2012e0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:47:13.026065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3b7dd1e5-72a0-4a1c-abde-c4f0ed2012e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:47:13.100344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 592/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:47:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:47:13.124418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222702d-eb32-48d7-83fb-dede417d9710 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:13.238837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222702d-eb32-48d7-83fb-dede417d9710 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:13.240703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:13.278355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf8dea-2e35-4710-9b71-7f273b11e564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:13.279087 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf8dea-2e35-4710-9b71-7f273b11e564 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:13.279366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf8dea-2e35-4710-9b71-7f273b11e564 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:13.279638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf8dea-2e35-4710-9b71-7f273b11e564 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:13.291025 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf8dea-2e35-4710-9b71-7f273b11e564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-01bf8dea-2e35-4710-9b71-7f273b11e564 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:47:13.298626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:47:13.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:47:13.300710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 603/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:47:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:47:13.306152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dd1e5-72a0-4a1c-abde-c4f0ed2012e0 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:13.363622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:13.418598 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dd1e5-72a0-4a1c-abde-c4f0ed2012e0 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:13.454306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e4c02-5a9f-4a36-ae96-89256f30cfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:13.455083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e4c02-5a9f-4a36-ae96-89256f30cfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:13.455358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e4c02-5a9f-4a36-ae96-89256f30cfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:13.455629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e4c02-5a9f-4a36-ae96-89256f30cfc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:13.456358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e4c02-5a9f-4a36-ae96-89256f30cfc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:47:13.456653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e4c02-5a9f-4a36-ae96-89256f30cfc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:13.458039 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e4c02-5a9f-4a36-ae96-89256f30cfc5 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:13.466890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 593/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:47:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:13.473694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf8dea-2e35-4710-9b71-7f273b11e564 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:13.576420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf8dea-2e35-4710-9b71-7f273b11e564 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:13.611687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41174e3-2f6e-4af0-82d1-dcf1fbb2fc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:13.612564 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41174e3-2f6e-4af0-82d1-dcf1fbb2fc99 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:13.612839 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41174e3-2f6e-4af0-82d1-dcf1fbb2fc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:13.613107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41174e3-2f6e-4af0-82d1-dcf1fbb2fc99 None admin] PATH_INFO: `/v3/users/37e082fb79ff4caf903d4c58ab9b09fe` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:13.614335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41174e3-2f6e-4af0-82d1-dcf1fbb2fc99 None admin] RBAC: Authorizing `identity:get_user(user_id=37e082fb79ff4caf903d4c58ab9b09fe)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:13.615796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41174e3-2f6e-4af0-82d1-dcf1fbb2fc99 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:13.617513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 604/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 09:47:13 2022] GET /identity/v3/users/37e082fb79ff4caf903d4c58ab9b09fe => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:13.623589 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e4c02-5a9f-4a36-ae96-89256f30cfc5 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:13.728523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e4c02-5a9f-4a36-ae96-89256f30cfc5 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:13.739487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:13.764108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb08cfb-ac85-4269-821c-eecd22dfbfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:13.764766 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08cfb-ac85-4269-821c-eecd22dfbfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:13.765174 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08cfb-ac85-4269-821c-eecd22dfbfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:13.765525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08cfb-ac85-4269-821c-eecd22dfbfb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:13.835042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb08cfb-ac85-4269-821c-eecd22dfbfb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:13.836411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb08cfb-ac85-4269-821c-eecd22dfbfb3 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:13.937542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 594/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:47:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:47:13.943911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41174e3-2f6e-4af0-82d1-dcf1fbb2fc99 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:13.986965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:14.067122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41174e3-2f6e-4af0-82d1-dcf1fbb2fc99 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:14.102852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677aa691-ccac-4e34-a150-c3868a082c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:14.103628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aa691-ccac-4e34-a150-c3868a082c41 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:14.103901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aa691-ccac-4e34-a150-c3868a082c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:14.104172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677aa691-ccac-4e34-a150-c3868a082c41 None admin] PATH_INFO: `/v3/projects/c17761004e3b4de3bba61f4163288439` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:14.105424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677aa691-ccac-4e34-a150-c3868a082c41 None admin] RBAC: Authorizing `identity:get_project(project_id=c17761004e3b4de3bba61f4163288439)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:14.106926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677aa691-ccac-4e34-a150-c3868a082c41 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:14.108674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 605/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 09:47:13 2022] GET /identity/v3/projects/c17761004e3b4de3bba61f4163288439 => generated 361 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:14.115166 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb08cfb-ac85-4269-821c-eecd22dfbfb3 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:47:14.128904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:14.218855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb08cfb-ac85-4269-821c-eecd22dfbfb3 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:47:14.256222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b812b0d-6d5a-4d33-892e-30b2bfae9575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:47:14.256889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b812b0d-6d5a-4d33-892e-30b2bfae9575 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:47:14.257171 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b812b0d-6d5a-4d33-892e-30b2bfae9575 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:47:14.257440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b812b0d-6d5a-4d33-892e-30b2bfae9575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:47:14.258280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b812b0d-6d5a-4d33-892e-30b2bfae9575 None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=c17761004e3b4de3bba61f4163288439, user.id=37e082fb79ff4caf903d4c58ab9b09fe) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:47:14.258608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b812b0d-6d5a-4d33-892e-30b2bfae9575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:47:14.260074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b812b0d-6d5a-4d33-892e-30b2bfae9575 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:47:14.266791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 595/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:47:14 2022] GET /identity/v3/role_assignments?user.id=37e082fb79ff4caf903d4c58ab9b09fe&scope.project.id=c17761004e3b4de3bba61f4163288439&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:47:24.097885 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=51 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:47:24.263640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=63 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:13.166389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a331d-66b3-48f1-853a-3f25b47a0b77 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:13.167259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a331d-66b3-48f1-853a-3f25b47a0b77 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:13.167259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a331d-66b3-48f1-853a-3f25b47a0b77 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:13.168831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 606/1201] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:48:13.176212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9f0cf-e72a-4fdb-ba31-654df208baba None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:13.177098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9f0cf-e72a-4fdb-ba31-654df208baba None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:13.177351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9f0cf-e72a-4fdb-ba31-654df208baba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:13.201104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fee9f0cf-e72a-4fdb-ba31-654df208baba None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:48:13.203243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fee9f0cf-e72a-4fdb-ba31-654df208baba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:48:13.329093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 596/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:48:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:48:13.356812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a331d-66b3-48f1-853a-3f25b47a0b77 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:13.579889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a331d-66b3-48f1-853a-3f25b47a0b77 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:13.645829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733690-2db9-4759-8626-f513d4ade6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:13.647289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733690-2db9-4759-8626-f513d4ade6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:13.647833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733690-2db9-4759-8626-f513d4ade6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:13.648270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733690-2db9-4759-8626-f513d4ade6d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:13.676668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32733690-2db9-4759-8626-f513d4ade6d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:13.680746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32733690-2db9-4759-8626-f513d4ade6d5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-32733690-2db9-4759-8626-f513d4ade6d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:13.716453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:13.719940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:13.722794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:13.722794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:13.722794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:13.722794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:13.722794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 607/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:13.726613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9f0cf-e72a-4fdb-ba31-654df208baba None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:13.933825 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9f0cf-e72a-4fdb-ba31-654df208baba None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:13.987899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:13.996597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76307230-aaae-4d57-bc54-67006d7b2207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:13.997410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76307230-aaae-4d57-bc54-67006d7b2207 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:13.997808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76307230-aaae-4d57-bc54-67006d7b2207 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:13.998270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76307230-aaae-4d57-bc54-67006d7b2207 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:13.999426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76307230-aaae-4d57-bc54-67006d7b2207 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:13.999937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76307230-aaae-4d57-bc54-67006d7b2207 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:14.001434 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76307230-aaae-4d57-bc54-67006d7b2207 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:14.022727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 597/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:14.024451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733690-2db9-4759-8626-f513d4ade6d5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:14.145002 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:14.235012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733690-2db9-4759-8626-f513d4ade6d5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:14.309211 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc691fe-e529-4cb6-88cb-161d97c35888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:14.310504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc691fe-e529-4cb6-88cb-161d97c35888 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:14.311194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc691fe-e529-4cb6-88cb-161d97c35888 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:14.311687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc691fe-e529-4cb6-88cb-161d97c35888 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:14.313237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc691fe-e529-4cb6-88cb-161d97c35888 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:14.316321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc691fe-e529-4cb6-88cb-161d97c35888 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:14.366628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 608/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 09:48:14 2022] POST /identity/v3/users => generated 251 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:48:16.969759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df5ed2-9803-41d3-83ed-ee6d99557f80 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:16.970642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df5ed2-9803-41d3-83ed-ee6d99557f80 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:16.970642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df5ed2-9803-41d3-83ed-ee6d99557f80 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:16.973107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 598/1206] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:48:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:48:16.979688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996330cc-d010-48f9-afa5-7188a01a2905 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:16.980538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996330cc-d010-48f9-afa5-7188a01a2905 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:16.980538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996330cc-d010-48f9-afa5-7188a01a2905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:17.000878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-996330cc-d010-48f9-afa5-7188a01a2905 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:48:17.002975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-996330cc-d010-48f9-afa5-7188a01a2905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:48:17.132535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 609/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:48:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:48:17.174843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df5ed2-9803-41d3-83ed-ee6d99557f80 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:17.380140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df5ed2-9803-41d3-83ed-ee6d99557f80 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:17.441141 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdee5b9-99e1-4aa6-b601-40b6a7a33825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:17.442260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdee5b9-99e1-4aa6-b601-40b6a7a33825 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:17.442715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdee5b9-99e1-4aa6-b601-40b6a7a33825 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:17.443179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdee5b9-99e1-4aa6-b601-40b6a7a33825 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:17.460129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdee5b9-99e1-4aa6-b601-40b6a7a33825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-cfdee5b9-99e1-4aa6-b601-40b6a7a33825 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:48:17.471828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:17.474002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:48:17.476042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:17.479210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 599/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:48:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:17.484802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996330cc-d010-48f9-afa5-7188a01a2905 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:17.557590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:17.671834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996330cc-d010-48f9-afa5-7188a01a2905 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:17.749649 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bcd2eb-6362-4ec4-b4d4-3e46451bdf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:17.750919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcd2eb-6362-4ec4-b4d4-3e46451bdf59 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:17.751404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcd2eb-6362-4ec4-b4d4-3e46451bdf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:17.751873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcd2eb-6362-4ec4-b4d4-3e46451bdf59 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:17.753606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bcd2eb-6362-4ec4-b4d4-3e46451bdf59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:17.754240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bcd2eb-6362-4ec4-b4d4-3e46451bdf59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:17.757676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bcd2eb-6362-4ec4-b4d4-3e46451bdf59 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:17.772140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 610/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:48:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:17.783580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdee5b9-99e1-4aa6-b601-40b6a7a33825 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:17.896009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:17.966084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdee5b9-99e1-4aa6-b601-40b6a7a33825 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:18.014076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35d454-bb4a-472f-a8df-54b62e9a4b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:18.015932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35d454-bb4a-472f-a8df-54b62e9a4b85 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:18.016387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35d454-bb4a-472f-a8df-54b62e9a4b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:18.016983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35d454-bb4a-472f-a8df-54b62e9a4b85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:18.033055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35d454-bb4a-472f-a8df-54b62e9a4b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:18.035254 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35d454-bb4a-472f-a8df-54b62e9a4b85 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-5e35d454-bb4a-472f-a8df-54b62e9a4b85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:18.062950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:18.065102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:18.066203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:18.066203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:18.066203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:18.066203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:18.066203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 600/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:18.075063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bcd2eb-6362-4ec4-b4d4-3e46451bdf59 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:18.263658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:18.271629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bcd2eb-6362-4ec4-b4d4-3e46451bdf59 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:18.326259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465afb2b-6399-42f3-b312-f7029bcc724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:18.327479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465afb2b-6399-42f3-b312-f7029bcc724e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:18.327900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465afb2b-6399-42f3-b312-f7029bcc724e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:18.328333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465afb2b-6399-42f3-b312-f7029bcc724e None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:18.329473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465afb2b-6399-42f3-b312-f7029bcc724e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:18.330004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465afb2b-6399-42f3-b312-f7029bcc724e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:18.331911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465afb2b-6399-42f3-b312-f7029bcc724e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:18.356212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 611/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:18.362426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35d454-bb4a-472f-a8df-54b62e9a4b85 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:18.525651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35d454-bb4a-472f-a8df-54b62e9a4b85 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:18.528066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:18.582485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23788659-7f7e-44de-a07d-fda6dbeff05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:18.583673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23788659-7f7e-44de-a07d-fda6dbeff05a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:18.584288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23788659-7f7e-44de-a07d-fda6dbeff05a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:18.584817 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23788659-7f7e-44de-a07d-fda6dbeff05a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:18.596949 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23788659-7f7e-44de-a07d-fda6dbeff05a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:18.598719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23788659-7f7e-44de-a07d-fda6dbeff05a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-23788659-7f7e-44de-a07d-fda6dbeff05a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:48:18.616000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:18.617300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:18.618493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:18.618493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:18.618493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:48:18.618493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:48:18.618493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:48:18.618493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:48:18.618493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:48:18.618493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:48:18.618493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:18.618493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:18.620102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 601/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:18.626943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465afb2b-6399-42f3-b312-f7029bcc724e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:18.790388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465afb2b-6399-42f3-b312-f7029bcc724e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:18.856750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91020f-2cdb-4e57-b496-553c10a07d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:18.858040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91020f-2cdb-4e57-b496-553c10a07d1f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:18.858484 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91020f-2cdb-4e57-b496-553c10a07d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:18.858838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91020f-2cdb-4e57-b496-553c10a07d1f None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:18.860186 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91020f-2cdb-4e57-b496-553c10a07d1f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:18.860699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91020f-2cdb-4e57-b496-553c10a07d1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:18.863150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91020f-2cdb-4e57-b496-553c10a07d1f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:18.893642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 612/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 09:48:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:48:18.902202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23788659-7f7e-44de-a07d-fda6dbeff05a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:19.060791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23788659-7f7e-44de-a07d-fda6dbeff05a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:19.113598 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bde90b-3575-4100-8d03-889fb3861f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:19.114251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bde90b-3575-4100-8d03-889fb3861f0e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:19.114734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bde90b-3575-4100-8d03-889fb3861f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:19.116668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bde90b-3575-4100-8d03-889fb3861f0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:19.153308 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:19.204739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bde90b-3575-4100-8d03-889fb3861f0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:19.206315 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bde90b-3575-4100-8d03-889fb3861f0e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:19.344896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 602/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:48:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 09:48:19.352036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91020f-2cdb-4e57-b496-553c10a07d1f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:19.388924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:19.501046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91020f-2cdb-4e57-b496-553c10a07d1f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:19.550308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6831b-fbef-4e96-8402-3d72555dd4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:19.551749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6831b-fbef-4e96-8402-3d72555dd4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:19.552337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6831b-fbef-4e96-8402-3d72555dd4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:19.552707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6831b-fbef-4e96-8402-3d72555dd4e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:19.571149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6831b-fbef-4e96-8402-3d72555dd4e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:19.572693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6831b-fbef-4e96-8402-3d72555dd4e2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-2ff6831b-fbef-4e96-8402-3d72555dd4e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:19.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:19.599932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:19.601026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:19.601026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:19.601026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:19.601026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:19.601026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 613/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:48:19.604572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bde90b-3575-4100-8d03-889fb3861f0e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:19.686741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:19.746345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bde90b-3575-4100-8d03-889fb3861f0e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:19.799288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bde5e2-2988-4e73-aa76-a53c2d8f2345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:19.801080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bde5e2-2988-4e73-aa76-a53c2d8f2345 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:19.801499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bde5e2-2988-4e73-aa76-a53c2d8f2345 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:19.801795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bde5e2-2988-4e73-aa76-a53c2d8f2345 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:19.802729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bde5e2-2988-4e73-aa76-a53c2d8f2345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:19.803022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bde5e2-2988-4e73-aa76-a53c2d8f2345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:19.804684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bde5e2-2988-4e73-aa76-a53c2d8f2345 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:19.819683 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 603/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:48:19.826194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6831b-fbef-4e96-8402-3d72555dd4e2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:19.937883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:19.991574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6831b-fbef-4e96-8402-3d72555dd4e2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:20.071220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca9c4d-29f8-49f8-8739-320e53d70796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:20.072218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca9c4d-29f8-49f8-8739-320e53d70796 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:20.072681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca9c4d-29f8-49f8-8739-320e53d70796 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:20.073113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca9c4d-29f8-49f8-8739-320e53d70796 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:20.083356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca9c4d-29f8-49f8-8739-320e53d70796 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:20.085058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca9c4d-29f8-49f8-8739-320e53d70796 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-dfca9c4d-29f8-49f8-8739-320e53d70796 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:48:20.110967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:20.112161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:20.113244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:48:20.113244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:48:20.113244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:48:20.113244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:48:20.113244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:48:20.113244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:48:20.113244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:20.113244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:20.113825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 614/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:48:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:20.127128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bde5e2-2988-4e73-aa76-a53c2d8f2345 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:20.279268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bde5e2-2988-4e73-aa76-a53c2d8f2345 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:20.310730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:20.325718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0987c4-ee02-41f2-b6f3-08cc126354e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:20.326518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0987c4-ee02-41f2-b6f3-08cc126354e0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:20.326848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0987c4-ee02-41f2-b6f3-08cc126354e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:20.327119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0987c4-ee02-41f2-b6f3-08cc126354e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:20.328117 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0987c4-ee02-41f2-b6f3-08cc126354e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:20.328404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0987c4-ee02-41f2-b6f3-08cc126354e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:20.329997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0987c4-ee02-41f2-b6f3-08cc126354e0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:20.347632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 604/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:48:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:20.355084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca9c4d-29f8-49f8-8739-320e53d70796 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:20.511553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca9c4d-29f8-49f8-8739-320e53d70796 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:20.553655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:20.561190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955834a-1687-4023-9dc0-60095aae8630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:20.562742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955834a-1687-4023-9dc0-60095aae8630 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:20.562742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955834a-1687-4023-9dc0-60095aae8630 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:20.562924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955834a-1687-4023-9dc0-60095aae8630 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:20.563922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955834a-1687-4023-9dc0-60095aae8630 None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=e9bff6c855ea46feaa5526d2c27c9026) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:20.564275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955834a-1687-4023-9dc0-60095aae8630 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:20.566151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955834a-1687-4023-9dc0-60095aae8630 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:20.582593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 615/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:48:20 2022] GET /identity/v3/role_assignments?user.id=e9bff6c855ea46feaa5526d2c27c9026&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:48:22.731247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8fee-fe94-4b33-8ed7-703ffc84c4ff None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:22.731850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8fee-fe94-4b33-8ed7-703ffc84c4ff None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:22.732564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8fee-fe94-4b33-8ed7-703ffc84c4ff None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:22.735469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 605/1220] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:48:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:48:22.741390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264c0b9-6b84-4767-a1d7-01479f707e50 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:22.741851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264c0b9-6b84-4767-a1d7-01479f707e50 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:22.742003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264c0b9-6b84-4767-a1d7-01479f707e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:22.763475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e264c0b9-6b84-4767-a1d7-01479f707e50 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:48:22.765488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e264c0b9-6b84-4767-a1d7-01479f707e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:48:22.861447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 616/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:48:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:48:22.885682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e8fee-fe94-4b33-8ed7-703ffc84c4ff None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:23.032790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e8fee-fe94-4b33-8ed7-703ffc84c4ff None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:23.083294 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc747bc4-1b66-4f10-846d-815eec6710f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:23.084078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc747bc4-1b66-4f10-846d-815eec6710f1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:23.084356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc747bc4-1b66-4f10-846d-815eec6710f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:23.084623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc747bc4-1b66-4f10-846d-815eec6710f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:23.099680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc747bc4-1b66-4f10-846d-815eec6710f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-dc747bc4-1b66-4f10-846d-815eec6710f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:48:23.107469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:23.109363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:23.110269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 606/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:48:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:23.117821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264c0b9-6b84-4767-a1d7-01479f707e50 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:23.243872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264c0b9-6b84-4767-a1d7-01479f707e50 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:23.273355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:23.288712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d535208-f8eb-4203-a2c4-10173e769b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:23.289438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535208-f8eb-4203-a2c4-10173e769b23 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:23.289895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535208-f8eb-4203-a2c4-10173e769b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:23.290348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535208-f8eb-4203-a2c4-10173e769b23 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:23.291280 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d535208-f8eb-4203-a2c4-10173e769b23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:23.291626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d535208-f8eb-4203-a2c4-10173e769b23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:23.293237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d535208-f8eb-4203-a2c4-10173e769b23 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:23.304773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 617/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:48:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:23.311144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc747bc4-1b66-4f10-846d-815eec6710f1 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:23.375202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:23.472512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc747bc4-1b66-4f10-846d-815eec6710f1 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:23.535677 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca778f7-54a8-412d-8a6c-36186328e5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:23.537279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca778f7-54a8-412d-8a6c-36186328e5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:23.537775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca778f7-54a8-412d-8a6c-36186328e5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:23.538249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca778f7-54a8-412d-8a6c-36186328e5c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:23.559214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca778f7-54a8-412d-8a6c-36186328e5c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:23.562715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca778f7-54a8-412d-8a6c-36186328e5c2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-aca778f7-54a8-412d-8a6c-36186328e5c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:23.591159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:23.593036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:23.594853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:23.594853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:23.594853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:23.594853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:23.594853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 607/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:48:23.601433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d535208-f8eb-4203-a2c4-10173e769b23 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:23.751015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d535208-f8eb-4203-a2c4-10173e769b23 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:23.803217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aa6ca-e8b4-4d15-af2a-3cbf0c36cc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:23.803918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa6ca-e8b4-4d15-af2a-3cbf0c36cc90 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:23.804196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa6ca-e8b4-4d15-af2a-3cbf0c36cc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:23.804466 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa6ca-e8b4-4d15-af2a-3cbf0c36cc90 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:23.805227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6aa6ca-e8b4-4d15-af2a-3cbf0c36cc90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:23.805535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6aa6ca-e8b4-4d15-af2a-3cbf0c36cc90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:23.807061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6aa6ca-e8b4-4d15-af2a-3cbf0c36cc90 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:23.825541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 618/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:48:23.838097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca778f7-54a8-412d-8a6c-36186328e5c2 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:24.005231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:24.020356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca778f7-54a8-412d-8a6c-36186328e5c2 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:24.083467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7655134-3777-4f45-afb9-7dbe44c9522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:24.084746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7655134-3777-4f45-afb9-7dbe44c9522a None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:24.085385 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7655134-3777-4f45-afb9-7dbe44c9522a None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:24.085896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7655134-3777-4f45-afb9-7dbe44c9522a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:24.104125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7655134-3777-4f45-afb9-7dbe44c9522a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:24.107614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7655134-3777-4f45-afb9-7dbe44c9522a None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-d7655134-3777-4f45-afb9-7dbe44c9522a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:48:24.131710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:24.133513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:24.135403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:24.135403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:24.135403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:48:24.135403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:48:24.135403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:48:24.135403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:48:24.135403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:48:24.135403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:48:24.135403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:24.135403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:24.135403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 608/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:24.140648 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aa6ca-e8b4-4d15-af2a-3cbf0c36cc90 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:24.200940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:24.270823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aa6ca-e8b4-4d15-af2a-3cbf0c36cc90 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:24.318896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bf434-56d6-4b17-8917-9daaa5dcec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:24.319795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bf434-56d6-4b17-8917-9daaa5dcec7c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:24.320243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bf434-56d6-4b17-8917-9daaa5dcec7c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:24.320697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bf434-56d6-4b17-8917-9daaa5dcec7c None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:24.321791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bf434-56d6-4b17-8917-9daaa5dcec7c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:24.322205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bf434-56d6-4b17-8917-9daaa5dcec7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:24.323917 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bf434-56d6-4b17-8917-9daaa5dcec7c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:24.350210 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 619/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 09:48:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:24.357603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7655134-3777-4f45-afb9-7dbe44c9522a None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:24.512258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7655134-3777-4f45-afb9-7dbe44c9522a None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:24.563614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2abd6-5091-4f3e-8a7b-fd6e978830d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:24.564857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2abd6-5091-4f3e-8a7b-fd6e978830d5 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:24.565304 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2abd6-5091-4f3e-8a7b-fd6e978830d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:24.565710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2abd6-5091-4f3e-8a7b-fd6e978830d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:24.582493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2abd6-5091-4f3e-8a7b-fd6e978830d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:24.585011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2abd6-5091-4f3e-8a7b-fd6e978830d5 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-07f2abd6-5091-4f3e-8a7b-fd6e978830d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:24.613956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:24.615663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:24.617526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:24.617526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:24.617526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:24.617526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:24.617526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 609/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:24.625901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bf434-56d6-4b17-8917-9daaa5dcec7c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:24.783039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bf434-56d6-4b17-8917-9daaa5dcec7c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:24.786062 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:24.832948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b95f04-e616-4741-95a4-001c65902882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:24.833914 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b95f04-e616-4741-95a4-001c65902882 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:24.834325 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b95f04-e616-4741-95a4-001c65902882 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:24.834868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b95f04-e616-4741-95a4-001c65902882 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:24.835699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b95f04-e616-4741-95a4-001c65902882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:24.836129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b95f04-e616-4741-95a4-001c65902882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:24.837898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b95f04-e616-4741-95a4-001c65902882 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:24.855287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 620/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:24.862935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2abd6-5091-4f3e-8a7b-fd6e978830d5 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:24.874428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:25.037634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2abd6-5091-4f3e-8a7b-fd6e978830d5 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:25.097575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d49d0-bb59-4660-8600-bd3daec8784e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:25.099050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d49d0-bb59-4660-8600-bd3daec8784e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:25.099536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d49d0-bb59-4660-8600-bd3daec8784e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:25.100032 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d49d0-bb59-4660-8600-bd3daec8784e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:25.114606 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d49d0-bb59-4660-8600-bd3daec8784e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:25.117472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d49d0-bb59-4660-8600-bd3daec8784e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-fa3d49d0-bb59-4660-8600-bd3daec8784e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:48:25.147074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:25.149074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:25.151246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:48:25.151246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:48:25.151246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:48:25.151246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:48:25.151246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:48:25.151246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:48:25.151246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:25.151246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:25.154806 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 610/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:48:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:25.158717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b95f04-e616-4741-95a4-001c65902882 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:25.329474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b95f04-e616-4741-95a4-001c65902882 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:25.378354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d2627-2d1f-42c6-9dee-6efd55bafd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:25.379194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d2627-2d1f-42c6-9dee-6efd55bafd92 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:25.379591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d2627-2d1f-42c6-9dee-6efd55bafd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:25.379966 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d2627-2d1f-42c6-9dee-6efd55bafd92 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:25.380869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d2627-2d1f-42c6-9dee-6efd55bafd92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:25.381261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d2627-2d1f-42c6-9dee-6efd55bafd92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:25.382957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d2627-2d1f-42c6-9dee-6efd55bafd92 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:25.398341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 621/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:48:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:48:25.410786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d49d0-bb59-4660-8600-bd3daec8784e None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:25.508957 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:25.566982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d49d0-bb59-4660-8600-bd3daec8784e None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:25.634499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f4af2-3a16-482f-ab84-58fe62a48c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:25.636290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f4af2-3a16-482f-ab84-58fe62a48c81 None admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:25.636850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f4af2-3a16-482f-ab84-58fe62a48c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:25.637374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f4af2-3a16-482f-ab84-58fe62a48c81 None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/e9bff6c855ea46feaa5526d2c27c9026/roles/9b6eb8ccaac643cc93781d6abe31bb8f` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:25.666674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f4af2-3a16-482f-ab84-58fe62a48c81 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=e9bff6c855ea46feaa5526d2c27c9026, role_id=9b6eb8ccaac643cc93781d6abe31bb8f)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:25.669038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f4af2-3a16-482f-ab84-58fe62a48c81 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:25.689236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 611/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:48:25 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/e9bff6c855ea46feaa5526d2c27c9026/roles/9b6eb8ccaac643cc93781d6abe31bb8f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:48:27.953463 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97d928-86ab-4742-bb8d-2ebf71f2ec65 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:27.954134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97d928-86ab-4742-bb8d-2ebf71f2ec65 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:27.954391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97d928-86ab-4742-bb8d-2ebf71f2ec65 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:27.957022 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 622/1233] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:48:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:48:27.964234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e499e02-7225-45fd-86cc-a6cb3d3a23a5 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:27.964835 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e499e02-7225-45fd-86cc-a6cb3d3a23a5 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:27.964929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e499e02-7225-45fd-86cc-a6cb3d3a23a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:27.984198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6e499e02-7225-45fd-86cc-a6cb3d3a23a5 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:48:27.985534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6e499e02-7225-45fd-86cc-a6cb3d3a23a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:48:28.103122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 612/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:48:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:48:28.142195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97d928-86ab-4742-bb8d-2ebf71f2ec65 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:28.189273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:28.349277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97d928-86ab-4742-bb8d-2ebf71f2ec65 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:28.438406 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e2f7f-d657-49f6-b323-1e4955d1b65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:28.441411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e2f7f-d657-49f6-b323-1e4955d1b65b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:28.441411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e2f7f-d657-49f6-b323-1e4955d1b65b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:28.441411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e2f7f-d657-49f6-b323-1e4955d1b65b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:28.462000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e2f7f-d657-49f6-b323-1e4955d1b65b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-6a6e2f7f-d657-49f6-b323-1e4955d1b65b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:48:28.470939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:28.473249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:28.475393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 623/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 09:48:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:28.481169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e499e02-7225-45fd-86cc-a6cb3d3a23a5 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:28.547040 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:28.656887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e499e02-7225-45fd-86cc-a6cb3d3a23a5 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:28.717317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ab9ea-bb85-454c-b81d-d96bccf8c871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:28.718303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ab9ea-bb85-454c-b81d-d96bccf8c871 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:28.718750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ab9ea-bb85-454c-b81d-d96bccf8c871 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:28.719149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ab9ea-bb85-454c-b81d-d96bccf8c871 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:28.720208 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ab9ea-bb85-454c-b81d-d96bccf8c871 None admin] RBAC: Adding query filter params (name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:28.720645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ab9ea-bb85-454c-b81d-d96bccf8c871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:28.723017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ab9ea-bb85-454c-b81d-d96bccf8c871 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:28.736006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 613/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 09:48:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:48:28.742681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e2f7f-d657-49f6-b323-1e4955d1b65b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:28.890553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:28.914521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e2f7f-d657-49f6-b323-1e4955d1b65b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:28.970183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac9f98-6b4b-4e3c-baaa-42fa07a67d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:28.971945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac9f98-6b4b-4e3c-baaa-42fa07a67d71 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:28.972424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac9f98-6b4b-4e3c-baaa-42fa07a67d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:28.972902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac9f98-6b4b-4e3c-baaa-42fa07a67d71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:28.995225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac9f98-6b4b-4e3c-baaa-42fa07a67d71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:28.997195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac9f98-6b4b-4e3c-baaa-42fa07a67d71 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-33ac9f98-6b4b-4e3c-baaa-42fa07a67d71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:29.035180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:29.036602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:29.037844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:29.037844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:29.037844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:29.037844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:29.037844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 624/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:29.047959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ab9ea-bb85-454c-b81d-d96bccf8c871 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:29.206761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ab9ea-bb85-454c-b81d-d96bccf8c871 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:29.220933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:29.263295 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c65d2-3cff-475e-85ec-501422c15d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:29.264543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c65d2-3cff-475e-85ec-501422c15d44 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:29.264818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c65d2-3cff-475e-85ec-501422c15d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:29.265152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c65d2-3cff-475e-85ec-501422c15d44 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:29.266738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c65d2-3cff-475e-85ec-501422c15d44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:29.267057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c65d2-3cff-475e-85ec-501422c15d44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:29.268867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c65d2-3cff-475e-85ec-501422c15d44 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:29.284633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 614/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:29.292395 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac9f98-6b4b-4e3c-baaa-42fa07a67d71 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:29.449267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac9f98-6b4b-4e3c-baaa-42fa07a67d71 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:29.518709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6bc8e-cb99-4951-ac02-d18c1f1a6af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:29.520004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6bc8e-cb99-4951-ac02-d18c1f1a6af2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:29.520468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6bc8e-cb99-4951-ac02-d18c1f1a6af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:29.520943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6bc8e-cb99-4951-ac02-d18c1f1a6af2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:29.526700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:29.538885 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6bc8e-cb99-4951-ac02-d18c1f1a6af2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:29.540817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6bc8e-cb99-4951-ac02-d18c1f1a6af2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-52e6bc8e-cb99-4951-ac02-d18c1f1a6af2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:48:29.558632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:29.560537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:29.562739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:29.562739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:29.562739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:48:29.562739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:48:29.562739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:48:29.562739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:48:29.562739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:48:29.562739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:48:29.562739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:29.562739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:29.562739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 625/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:29.566709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c65d2-3cff-475e-85ec-501422c15d44 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:29.723488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c65d2-3cff-475e-85ec-501422c15d44 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:29.787182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251bdbe-ecac-4072-8846-abd0c44becec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:29.788302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251bdbe-ecac-4072-8846-abd0c44becec None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:29.788774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251bdbe-ecac-4072-8846-abd0c44becec None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:29.789221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251bdbe-ecac-4072-8846-abd0c44becec None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:29.790827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251bdbe-ecac-4072-8846-abd0c44becec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:29.791526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251bdbe-ecac-4072-8846-abd0c44becec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:29.794068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251bdbe-ecac-4072-8846-abd0c44becec None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:29.836049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 615/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 09:48:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:29.843047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6bc8e-cb99-4951-ac02-d18c1f1a6af2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:29.981571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6bc8e-cb99-4951-ac02-d18c1f1a6af2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:30.032165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c448f3f-fa62-46bf-bb7f-38614a703f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:30.033146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c448f3f-fa62-46bf-bb7f-38614a703f0e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:30.033570 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c448f3f-fa62-46bf-bb7f-38614a703f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:30.034001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c448f3f-fa62-46bf-bb7f-38614a703f0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:30.121738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:30.131910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c448f3f-fa62-46bf-bb7f-38614a703f0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:30.133982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c448f3f-fa62-46bf-bb7f-38614a703f0e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:30.247634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 626/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:48:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:48:30.254011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251bdbe-ecac-4072-8846-abd0c44becec None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:30.286965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:30.375348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251bdbe-ecac-4072-8846-abd0c44becec None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:30.424904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae12df1-9832-4d0a-af78-fb32adf6d10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:30.426249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae12df1-9832-4d0a-af78-fb32adf6d10d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:30.427379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae12df1-9832-4d0a-af78-fb32adf6d10d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:30.427749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae12df1-9832-4d0a-af78-fb32adf6d10d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:30.448727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae12df1-9832-4d0a-af78-fb32adf6d10d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:30.451865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae12df1-9832-4d0a-af78-fb32adf6d10d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-4ae12df1-9832-4d0a-af78-fb32adf6d10d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:30.482743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:30.484878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:30.486643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:30.486643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:30.486643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:30.486643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:30.486643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 616/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:30.494073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c448f3f-fa62-46bf-bb7f-38614a703f0e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:30.636836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:30.647691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c448f3f-fa62-46bf-bb7f-38614a703f0e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:30.697228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc524f6-14fc-494e-851f-2b7ff9fb0cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:30.697917 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc524f6-14fc-494e-851f-2b7ff9fb0cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:30.698285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc524f6-14fc-494e-851f-2b7ff9fb0cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:30.698475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc524f6-14fc-494e-851f-2b7ff9fb0cf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:30.699252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc524f6-14fc-494e-851f-2b7ff9fb0cf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:30.699599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc524f6-14fc-494e-851f-2b7ff9fb0cf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:30.701065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc524f6-14fc-494e-851f-2b7ff9fb0cf3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:30.714367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 627/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:30.720900 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae12df1-9832-4d0a-af78-fb32adf6d10d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:30.832359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:30.859754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae12df1-9832-4d0a-af78-fb32adf6d10d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:30.913845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5052cb-39ca-47e4-a9fa-a5558d60b8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:30.915130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5052cb-39ca-47e4-a9fa-a5558d60b8ca None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:30.915604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5052cb-39ca-47e4-a9fa-a5558d60b8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:30.916065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5052cb-39ca-47e4-a9fa-a5558d60b8ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:30.925330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5052cb-39ca-47e4-a9fa-a5558d60b8ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:30.927023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5052cb-39ca-47e4-a9fa-a5558d60b8ca None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-ac5052cb-39ca-47e4-a9fa-a5558d60b8ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:48:30.951111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:30.952525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:30.953649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:48:30.953649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:48:30.953649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:48:30.953649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:48:30.953649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:48:30.953649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:48:30.953649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:30.953649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:30.959363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 617/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:48:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:30.961240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc524f6-14fc-494e-851f-2b7ff9fb0cf3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:31.108533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc524f6-14fc-494e-851f-2b7ff9fb0cf3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:31.151373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a193de6-f83c-4f0a-b8fb-cacea8dd166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:31.152151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a193de6-f83c-4f0a-b8fb-cacea8dd166e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:31.152549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a193de6-f83c-4f0a-b8fb-cacea8dd166e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:31.152879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a193de6-f83c-4f0a-b8fb-cacea8dd166e None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:31.153795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a193de6-f83c-4f0a-b8fb-cacea8dd166e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:31.154137 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a193de6-f83c-4f0a-b8fb-cacea8dd166e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:31.156118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a193de6-f83c-4f0a-b8fb-cacea8dd166e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:31.178907 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 628/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:48:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:31.180883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5052cb-39ca-47e4-a9fa-a5558d60b8ca None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:31.312014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5052cb-39ca-47e4-a9fa-a5558d60b8ca None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:31.356303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01de77-b168-4de2-8302-bf3119fd3f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:31.357231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01de77-b168-4de2-8302-bf3119fd3f1e None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:31.358044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01de77-b168-4de2-8302-bf3119fd3f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:31.358404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01de77-b168-4de2-8302-bf3119fd3f1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:31.359474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01de77-b168-4de2-8302-bf3119fd3f1e None admin] RBAC: Adding query filter params (role.id=9b6eb8ccaac643cc93781d6abe31bb8f, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=e9bff6c855ea46feaa5526d2c27c9026) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:31.360002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01de77-b168-4de2-8302-bf3119fd3f1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:31.362040 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01de77-b168-4de2-8302-bf3119fd3f1e None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:31.371177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 618/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:48:31 2022] GET /identity/v3/role_assignments?user.id=e9bff6c855ea46feaa5526d2c27c9026&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=9b6eb8ccaac643cc93781d6abe31bb8f => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:33.685799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d25610-20ea-4f6f-9b7e-cba242c162fd None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:33.686738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d25610-20ea-4f6f-9b7e-cba242c162fd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:33.686738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d25610-20ea-4f6f-9b7e-cba242c162fd None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:33.688180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 629/1247] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:48:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:48:33.696014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c14b1-20af-4f7e-870a-9c2550f777cb None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:33.696933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c14b1-20af-4f7e-870a-9c2550f777cb None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:33.697117 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c14b1-20af-4f7e-870a-9c2550f777cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:33.704451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:33.720340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-e03c14b1-20af-4f7e-870a-9c2550f777cb None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:48:33.722396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e03c14b1-20af-4f7e-870a-9c2550f777cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:48:33.851976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 619/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:48:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:48:33.884685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d25610-20ea-4f6f-9b7e-cba242c162fd None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:33.912845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:34.055304 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d25610-20ea-4f6f-9b7e-cba242c162fd None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:34.102185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cc75c-5d28-4bbb-9503-9ca006ffd183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:34.103278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cc75c-5d28-4bbb-9503-9ca006ffd183 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:34.103656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cc75c-5d28-4bbb-9503-9ca006ffd183 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:34.104080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cc75c-5d28-4bbb-9503-9ca006ffd183 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:34.121811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5cc75c-5d28-4bbb-9503-9ca006ffd183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-7e5cc75c-5d28-4bbb-9503-9ca006ffd183 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:48:34.132461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:34.133842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:48:34.134927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:34.136006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 630/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:48:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:34.142814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c14b1-20af-4f7e-870a-9c2550f777cb None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:34.292512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c14b1-20af-4f7e-870a-9c2550f777cb None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:34.345315 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:34.353642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e55b84-8a10-479f-a484-f1d9deaccc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:34.354461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e55b84-8a10-479f-a484-f1d9deaccc69 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:34.354950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e55b84-8a10-479f-a484-f1d9deaccc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:34.355412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e55b84-8a10-479f-a484-f1d9deaccc69 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:34.356797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e55b84-8a10-479f-a484-f1d9deaccc69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:34.357285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e55b84-8a10-479f-a484-f1d9deaccc69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:34.359921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e55b84-8a10-479f-a484-f1d9deaccc69 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:34.374893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 620/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:48:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:48:34.381540 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cc75c-5d28-4bbb-9503-9ca006ffd183 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:34.511185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:34.545758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cc75c-5d28-4bbb-9503-9ca006ffd183 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:34.605863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de6400-7cab-4476-87c7-acb489fc7399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:34.608104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de6400-7cab-4476-87c7-acb489fc7399 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:34.608104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de6400-7cab-4476-87c7-acb489fc7399 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:34.608104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de6400-7cab-4476-87c7-acb489fc7399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:34.623568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de6400-7cab-4476-87c7-acb489fc7399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:34.625239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de6400-7cab-4476-87c7-acb489fc7399 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-81de6400-7cab-4476-87c7-acb489fc7399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:34.646703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:34.648034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:34.650037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:34.650037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:34.650037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:34.650037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:34.650037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 631/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:34.657872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e55b84-8a10-479f-a484-f1d9deaccc69 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:34.797332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e55b84-8a10-479f-a484-f1d9deaccc69 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:34.873364 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef7641-aaae-4846-b018-68331556c5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:34.874524 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef7641-aaae-4846-b018-68331556c5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:34.875053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef7641-aaae-4846-b018-68331556c5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:34.875517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef7641-aaae-4846-b018-68331556c5d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:34.876638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef7641-aaae-4846-b018-68331556c5d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:34.877059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef7641-aaae-4846-b018-68331556c5d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:34.878930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef7641-aaae-4846-b018-68331556c5d3 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:34.898644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 621/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:34.906072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de6400-7cab-4476-87c7-acb489fc7399 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:35.078137 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de6400-7cab-4476-87c7-acb489fc7399 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:35.120091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:35.134514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a190e93-f29d-494f-8e04-0c76bba42b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:35.135810 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a190e93-f29d-494f-8e04-0c76bba42b41 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:35.136194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a190e93-f29d-494f-8e04-0c76bba42b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:35.136627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a190e93-f29d-494f-8e04-0c76bba42b41 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:35.150424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a190e93-f29d-494f-8e04-0c76bba42b41 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:35.153157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a190e93-f29d-494f-8e04-0c76bba42b41 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-3a190e93-f29d-494f-8e04-0c76bba42b41 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:48:35.177973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:35.180257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:35.181932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:35.181932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:35.181932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:48:35.181932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:48:35.181932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:48:35.181932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:48:35.181932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:48:35.181932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:48:35.181932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:35.181932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:35.181932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 632/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:35.187931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef7641-aaae-4846-b018-68331556c5d3 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:35.274810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:35.348586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef7641-aaae-4846-b018-68331556c5d3 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:35.407384 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3a821-7be0-48f3-a28d-71ea338c5fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:35.408139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3a821-7be0-48f3-a28d-71ea338c5fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:35.408523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3a821-7be0-48f3-a28d-71ea338c5fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:35.408981 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3a821-7be0-48f3-a28d-71ea338c5fa3 None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:35.410089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3a821-7be0-48f3-a28d-71ea338c5fa3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:35.410508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3a821-7be0-48f3-a28d-71ea338c5fa3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:35.412190 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3a821-7be0-48f3-a28d-71ea338c5fa3 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:35.440081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 622/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 09:48:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:48:35.448191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a190e93-f29d-494f-8e04-0c76bba42b41 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:35.609379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a190e93-f29d-494f-8e04-0c76bba42b41 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:35.653370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7dd61-cf95-41c8-9c4d-e14053c1e3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:35.654040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7dd61-cf95-41c8-9c4d-e14053c1e3ae None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:35.654411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7dd61-cf95-41c8-9c4d-e14053c1e3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:35.654771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7dd61-cf95-41c8-9c4d-e14053c1e3ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:35.756868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:35.780818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7dd61-cf95-41c8-9c4d-e14053c1e3ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:35.784095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7dd61-cf95-41c8-9c4d-e14053c1e3ae None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:35.941914 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 633/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:48:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:48:35.949776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3a821-7be0-48f3-a28d-71ea338c5fa3 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:36.115777 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3a821-7be0-48f3-a28d-71ea338c5fa3 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:36.141444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:36.180144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dfdca-ab98-4086-9f59-5e87602bc74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:36.181527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dfdca-ab98-4086-9f59-5e87602bc74f None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:36.182004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dfdca-ab98-4086-9f59-5e87602bc74f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:36.182465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dfdca-ab98-4086-9f59-5e87602bc74f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:36.201487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0dfdca-ab98-4086-9f59-5e87602bc74f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:36.204219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0dfdca-ab98-4086-9f59-5e87602bc74f None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-ab0dfdca-ab98-4086-9f59-5e87602bc74f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:36.239325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:36.241153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:36.242965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:36.242965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:36.242965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:36.242965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:36.242965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 623/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:36.249495 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7dd61-cf95-41c8-9c4d-e14053c1e3ae None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:36.411299 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:36.435220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7dd61-cf95-41c8-9c4d-e14053c1e3ae None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:36.487318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1416168-c3af-445c-be33-bfd350f8d670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:36.488026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1416168-c3af-445c-be33-bfd350f8d670 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:36.488303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1416168-c3af-445c-be33-bfd350f8d670 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:36.488597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1416168-c3af-445c-be33-bfd350f8d670 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:36.489350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1416168-c3af-445c-be33-bfd350f8d670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:36.489656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1416168-c3af-445c-be33-bfd350f8d670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:36.491143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1416168-c3af-445c-be33-bfd350f8d670 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:36.508923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 634/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:36.515864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dfdca-ab98-4086-9f59-5e87602bc74f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:36.664619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dfdca-ab98-4086-9f59-5e87602bc74f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:36.715217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2803634-fdd2-4b82-8d37-d270bc66e426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:36.716377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2803634-fdd2-4b82-8d37-d270bc66e426 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:36.716949 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2803634-fdd2-4b82-8d37-d270bc66e426 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:36.717396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2803634-fdd2-4b82-8d37-d270bc66e426 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:36.729471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2803634-fdd2-4b82-8d37-d270bc66e426 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:36.731519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2803634-fdd2-4b82-8d37-d270bc66e426 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-c2803634-fdd2-4b82-8d37-d270bc66e426 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:48:36.752489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:36.755441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:36.758071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:48:36.758071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:48:36.758071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:48:36.758071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:48:36.758071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:48:36.758071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:48:36.758071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:36.758071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:36.758071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 624/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:48:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:36.764942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1416168-c3af-445c-be33-bfd350f8d670 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:36.923527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1416168-c3af-445c-be33-bfd350f8d670 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:36.970831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10166c3-c91a-4b91-8090-a69c4cfe23b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:36.972354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10166c3-c91a-4b91-8090-a69c4cfe23b2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:36.972873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10166c3-c91a-4b91-8090-a69c4cfe23b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:36.973455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10166c3-c91a-4b91-8090-a69c4cfe23b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:36.975077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10166c3-c91a-4b91-8090-a69c4cfe23b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:36.975721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10166c3-c91a-4b91-8090-a69c4cfe23b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:36.978836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10166c3-c91a-4b91-8090-a69c4cfe23b2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:36.997593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 635/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:48:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:37.006410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2803634-fdd2-4b82-8d37-d270bc66e426 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:37.037438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:37.158013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2803634-fdd2-4b82-8d37-d270bc66e426 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:37.203790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3222ad-19df-42c2-b6c3-b39abb3807ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:37.204897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3222ad-19df-42c2-b6c3-b39abb3807ba None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:37.205355 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3222ad-19df-42c2-b6c3-b39abb3807ba None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:37.205825 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3222ad-19df-42c2-b6c3-b39abb3807ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:37.208933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3222ad-19df-42c2-b6c3-b39abb3807ba None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=e9bff6c855ea46feaa5526d2c27c9026) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:37.209480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3222ad-19df-42c2-b6c3-b39abb3807ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:37.212241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3222ad-19df-42c2-b6c3-b39abb3807ba None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:37.223538 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 625/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:48:37 2022] GET /identity/v3/role_assignments?user.id=e9bff6c855ea46feaa5526d2c27c9026&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:39.408652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4fac6-bdb6-4581-85af-39bcac4b570d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:39.409175 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4fac6-bdb6-4581-85af-39bcac4b570d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:39.409175 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4fac6-bdb6-4581-85af-39bcac4b570d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:39.410820 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 636/1261] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:48:39.417103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869ae2-9061-4c0f-9469-bf85d99f837e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:39.417920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869ae2-9061-4c0f-9469-bf85d99f837e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:39.417920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869ae2-9061-4c0f-9469-bf85d99f837e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:39.438035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b6869ae2-9061-4c0f-9469-bf85d99f837e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:48:39.439328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b6869ae2-9061-4c0f-9469-bf85d99f837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:48:39.526602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 626/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:48:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:48:39.557276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4fac6-bdb6-4581-85af-39bcac4b570d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:39.572983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:39.684154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4fac6-bdb6-4581-85af-39bcac4b570d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:39.757956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfab6cd-5dd8-4615-893c-a0b2d88f2866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:39.759102 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfab6cd-5dd8-4615-893c-a0b2d88f2866 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:39.759487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfab6cd-5dd8-4615-893c-a0b2d88f2866 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:39.759831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfab6cd-5dd8-4615-893c-a0b2d88f2866 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:39.782014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfab6cd-5dd8-4615-893c-a0b2d88f2866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-4cfab6cd-5dd8-4615-893c-a0b2d88f2866 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:48:39.789906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:39.792080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:39.793868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 637/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:48:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:39.802235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6869ae2-9061-4c0f-9469-bf85d99f837e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:39.828459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:39.939634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6869ae2-9061-4c0f-9469-bf85d99f837e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:39.981343 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840ffb4-2aa3-4ca1-b9d6-b47c0ea946b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:39.982072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ffb4-2aa3-4ca1-b9d6-b47c0ea946b0 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:39.982779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ffb4-2aa3-4ca1-b9d6-b47c0ea946b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:39.983086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ffb4-2aa3-4ca1-b9d6-b47c0ea946b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:39.984010 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840ffb4-2aa3-4ca1-b9d6-b47c0ea946b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:39.984321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840ffb4-2aa3-4ca1-b9d6-b47c0ea946b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:39.985834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840ffb4-2aa3-4ca1-b9d6-b47c0ea946b0 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:39.996301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 627/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:48:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:40.003273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfab6cd-5dd8-4615-893c-a0b2d88f2866 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:40.104509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:40.153935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfab6cd-5dd8-4615-893c-a0b2d88f2866 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:40.198705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f3796-cdfa-46b9-acc1-d01adbbd88cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:40.199441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f3796-cdfa-46b9-acc1-d01adbbd88cc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:40.199721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f3796-cdfa-46b9-acc1-d01adbbd88cc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:40.199955 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f3796-cdfa-46b9-acc1-d01adbbd88cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:40.213009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f3796-cdfa-46b9-acc1-d01adbbd88cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:40.214504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f3796-cdfa-46b9-acc1-d01adbbd88cc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-371f3796-cdfa-46b9-acc1-d01adbbd88cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:40.240512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:40.241753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:40.242797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:40.242797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:40.242797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:40.242797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:40.242797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 638/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:40.248847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840ffb4-2aa3-4ca1-b9d6-b47c0ea946b0 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:40.361000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:40.385063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840ffb4-2aa3-4ca1-b9d6-b47c0ea946b0 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:40.431851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3c0e5-5474-412f-98c9-44782ca384da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:40.432516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3c0e5-5474-412f-98c9-44782ca384da None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:40.432819 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3c0e5-5474-412f-98c9-44782ca384da None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:40.433083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3c0e5-5474-412f-98c9-44782ca384da None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:40.433810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3c0e5-5474-412f-98c9-44782ca384da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:40.434117 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3c0e5-5474-412f-98c9-44782ca384da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:40.435609 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3c0e5-5474-412f-98c9-44782ca384da None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:40.448642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 628/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:40.455150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f3796-cdfa-46b9-acc1-d01adbbd88cc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:40.574223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f3796-cdfa-46b9-acc1-d01adbbd88cc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:40.597851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:40.620598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9f14a-3fac-40bc-b514-6396c88ef5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:40.622098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9f14a-3fac-40bc-b514-6396c88ef5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:40.622378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9f14a-3fac-40bc-b514-6396c88ef5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:40.623328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9f14a-3fac-40bc-b514-6396c88ef5c2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:40.633873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9f14a-3fac-40bc-b514-6396c88ef5c2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:40.635407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9f14a-3fac-40bc-b514-6396c88ef5c2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f9e9f14a-3fac-40bc-b514-6396c88ef5c2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:48:40.650829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:40.652074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:40.653321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:40.653321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:40.653321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:48:40.653321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:48:40.653321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:48:40.653321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:48:40.653321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:48:40.653321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:48:40.653321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:40.653321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:40.653321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 639/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:40.658705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3c0e5-5474-412f-98c9-44782ca384da None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:40.781191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3c0e5-5474-412f-98c9-44782ca384da None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:40.826435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab418d-4d68-4cb3-900d-b13ccb4d2c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:40.827396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab418d-4d68-4cb3-900d-b13ccb4d2c4c None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:40.827714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab418d-4d68-4cb3-900d-b13ccb4d2c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:40.828028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab418d-4d68-4cb3-900d-b13ccb4d2c4c None admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:40.828930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab418d-4d68-4cb3-900d-b13ccb4d2c4c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:40.829280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab418d-4d68-4cb3-900d-b13ccb4d2c4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:40.830832 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab418d-4d68-4cb3-900d-b13ccb4d2c4c None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:40.841233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:40.852951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 629/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 09:48:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:40.859455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9f14a-3fac-40bc-b514-6396c88ef5c2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:40.967379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9f14a-3fac-40bc-b514-6396c88ef5c2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:41.004841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de657831-370d-4d54-8dd3-98059993a776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:41.005650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de657831-370d-4d54-8dd3-98059993a776 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:41.005929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de657831-370d-4d54-8dd3-98059993a776 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:41.006193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de657831-370d-4d54-8dd3-98059993a776 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:41.018422 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de657831-370d-4d54-8dd3-98059993a776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:41.019947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de657831-370d-4d54-8dd3-98059993a776 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-de657831-370d-4d54-8dd3-98059993a776 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:41.038506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:41.040430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:41.042332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:41.042332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:41.042332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:41.042332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:41.042332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 640/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:41.046547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab418d-4d68-4cb3-900d-b13ccb4d2c4c None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:41.152476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab418d-4d68-4cb3-900d-b13ccb4d2c4c None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:41.189876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df42436-2d0a-4797-b887-77c53abcfb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:41.190595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df42436-2d0a-4797-b887-77c53abcfb79 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:41.190862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df42436-2d0a-4797-b887-77c53abcfb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:41.191134 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df42436-2d0a-4797-b887-77c53abcfb79 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:41.191867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df42436-2d0a-4797-b887-77c53abcfb79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:41.192167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df42436-2d0a-4797-b887-77c53abcfb79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:41.193547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df42436-2d0a-4797-b887-77c53abcfb79 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:41.205834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 630/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:41.211838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de657831-370d-4d54-8dd3-98059993a776 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:41.251069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:41.354667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de657831-370d-4d54-8dd3-98059993a776 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:41.412558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7d1f0-f3a5-4ec2-b458-5183e9056fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:41.413863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7d1f0-f3a5-4ec2-b458-5183e9056fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:41.414307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7d1f0-f3a5-4ec2-b458-5183e9056fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:41.414811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7d1f0-f3a5-4ec2-b458-5183e9056fc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:41.427383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7d1f0-f3a5-4ec2-b458-5183e9056fc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:41.429882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7d1f0-f3a5-4ec2-b458-5183e9056fc7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-63d7d1f0-f3a5-4ec2-b458-5183e9056fc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:48:41.457429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:41.458965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:41.460086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:48:41.460086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:48:41.460086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:48:41.460086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:48:41.460086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:48:41.460086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:48:41.460086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:41.460086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:41.460086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 641/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:48:41.468657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df42436-2d0a-4797-b887-77c53abcfb79 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:41.542292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:41.594273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df42436-2d0a-4797-b887-77c53abcfb79 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:41.632508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8ab48-f0a0-4791-b5df-fbfb80b066ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:41.633083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8ab48-f0a0-4791-b5df-fbfb80b066ef None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:41.633359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8ab48-f0a0-4791-b5df-fbfb80b066ef None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:41.633631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8ab48-f0a0-4791-b5df-fbfb80b066ef None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:41.634408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8ab48-f0a0-4791-b5df-fbfb80b066ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:41.634722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8ab48-f0a0-4791-b5df-fbfb80b066ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:41.636197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8ab48-f0a0-4791-b5df-fbfb80b066ef None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:41.649264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 631/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:48:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:41.656109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7d1f0-f3a5-4ec2-b458-5183e9056fc7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:41.759268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:41.773236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7d1f0-f3a5-4ec2-b458-5183e9056fc7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:41.816527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee276eb6-ab65-4bc8-aa05-c6433ec1e192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:41.817534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee276eb6-ab65-4bc8-aa05-c6433ec1e192 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:41.817852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee276eb6-ab65-4bc8-aa05-c6433ec1e192 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:41.818165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee276eb6-ab65-4bc8-aa05-c6433ec1e192 None admin] PATH_INFO: `/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/e9bff6c855ea46feaa5526d2c27c9026/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:41.820198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee276eb6-ab65-4bc8-aa05-c6433ec1e192 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4c742f196174c239f6a9c4aba775ee4, user_id=e9bff6c855ea46feaa5526d2c27c9026, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:41.821698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee276eb6-ab65-4bc8-aa05-c6433ec1e192 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:41.835448 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 642/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 09:48:41 2022] PUT /identity/v3/projects/b4c742f196174c239f6a9c4aba775ee4/users/e9bff6c855ea46feaa5526d2c27c9026/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:48:43.901001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e259688-c4aa-45be-b691-006d4cc42e3d None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:43.901657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e259688-c4aa-45be-b691-006d4cc42e3d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:43.901657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e259688-c4aa-45be-b691-006d4cc42e3d None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:43.903061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 632/1274] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:48:43.910127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197fce8-d9ed-4fcd-a72e-43e36a925c19 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:43.910766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197fce8-d9ed-4fcd-a72e-43e36a925c19 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:43.910766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197fce8-d9ed-4fcd-a72e-43e36a925c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:43.929779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1197fce8-d9ed-4fcd-a72e-43e36a925c19 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:48:43.931059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1197fce8-d9ed-4fcd-a72e-43e36a925c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:48:44.012790 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 643/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:48:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:48:44.054195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e259688-c4aa-45be-b691-006d4cc42e3d None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:44.179627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e259688-c4aa-45be-b691-006d4cc42e3d None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:44.199802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:44.218995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b76d7f-e88a-4996-b13e-fe15eabcdf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:44.219809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b76d7f-e88a-4996-b13e-fe15eabcdf7f None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:44.220091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b76d7f-e88a-4996-b13e-fe15eabcdf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:44.220362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b76d7f-e88a-4996-b13e-fe15eabcdf7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:44.233001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b76d7f-e88a-4996-b13e-fe15eabcdf7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-e6b76d7f-e88a-4996-b13e-fe15eabcdf7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 09:48:44.240945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:44.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:44.243290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 633/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 09:48:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:44.248782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197fce8-d9ed-4fcd-a72e-43e36a925c19 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:44.382345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:44.419074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197fce8-d9ed-4fcd-a72e-43e36a925c19 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:44.490896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462dc60-2d5d-4110-9e14-45a68f503977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:44.500324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462dc60-2d5d-4110-9e14-45a68f503977 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:44.500881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462dc60-2d5d-4110-9e14-45a68f503977 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:44.501380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462dc60-2d5d-4110-9e14-45a68f503977 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:44.503024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462dc60-2d5d-4110-9e14-45a68f503977 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:44.503571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462dc60-2d5d-4110-9e14-45a68f503977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:44.506433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462dc60-2d5d-4110-9e14-45a68f503977 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:44.524417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 644/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 09:48:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:44.531020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b76d7f-e88a-4996-b13e-fe15eabcdf7f None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:44.691569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b76d7f-e88a-4996-b13e-fe15eabcdf7f None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:44.753946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9368df5-e88c-48ba-bbeb-729d3b6f014b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:44.755317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9368df5-e88c-48ba-bbeb-729d3b6f014b None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:44.755802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9368df5-e88c-48ba-bbeb-729d3b6f014b None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:44.756275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9368df5-e88c-48ba-bbeb-729d3b6f014b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:44.775823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9368df5-e88c-48ba-bbeb-729d3b6f014b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:44.777491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9368df5-e88c-48ba-bbeb-729d3b6f014b None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-e9368df5-e88c-48ba-bbeb-729d3b6f014b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:44.799503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:44.801336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:44.803131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:44.803131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:44.803131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:44.803131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:44.803131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 634/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:44.807480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462dc60-2d5d-4110-9e14-45a68f503977 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:44.930389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462dc60-2d5d-4110-9e14-45a68f503977 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:44.974742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce9408-13bd-4270-81fd-a66471f8d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:44.975446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9408-13bd-4270-81fd-a66471f8d1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:44.975724 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9408-13bd-4270-81fd-a66471f8d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:44.975999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9408-13bd-4270-81fd-a66471f8d1c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:44.976772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce9408-13bd-4270-81fd-a66471f8d1c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:44.977079 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce9408-13bd-4270-81fd-a66471f8d1c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:44.978531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce9408-13bd-4270-81fd-a66471f8d1c3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:44.992933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 645/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:44.999817 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9368df5-e88c-48ba-bbeb-729d3b6f014b None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:45.038643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:45.122861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9368df5-e88c-48ba-bbeb-729d3b6f014b None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:45.162785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cb899-163a-45ca-a7ae-36dfbe4b003d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:45.163662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb899-163a-45ca-a7ae-36dfbe4b003d None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:45.164028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb899-163a-45ca-a7ae-36dfbe4b003d None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:45.164377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb899-163a-45ca-a7ae-36dfbe4b003d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:45.173482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cb899-163a-45ca-a7ae-36dfbe4b003d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:45.175064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cb899-163a-45ca-a7ae-36dfbe4b003d None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-0d4cb899-163a-45ca-a7ae-36dfbe4b003d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 09:48:45.190594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:45.192426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:45.194401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:45.194401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:45.194401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 09:48:45.194401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 09:48:45.194401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 09:48:45.194401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 09:48:45.194401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 09:48:45.194401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 09:48:45.194401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 09:48:45.194401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:45.194401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 635/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:45.199584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce9408-13bd-4270-81fd-a66471f8d1c3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:45.204400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:45.349967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce9408-13bd-4270-81fd-a66471f8d1c3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:45.387582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fec99-bbfa-43f0-a5eb-1a1f584243c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:45.388408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fec99-bbfa-43f0-a5eb-1a1f584243c2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:45.388866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fec99-bbfa-43f0-a5eb-1a1f584243c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:45.389425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fec99-bbfa-43f0-a5eb-1a1f584243c2 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:45.390713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fec99-bbfa-43f0-a5eb-1a1f584243c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:45.391121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fec99-bbfa-43f0-a5eb-1a1f584243c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:45.392755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fec99-bbfa-43f0-a5eb-1a1f584243c2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:45.413981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 646/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 09:48:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:45.421303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cb899-163a-45ca-a7ae-36dfbe4b003d None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:45.534086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:45.547122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cb899-163a-45ca-a7ae-36dfbe4b003d None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:45.585790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d66e1-afcb-41cb-bc4d-a53af80daf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:45.586526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d66e1-afcb-41cb-bc4d-a53af80daf27 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:45.586915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d66e1-afcb-41cb-bc4d-a53af80daf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:45.587237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d66e1-afcb-41cb-bc4d-a53af80daf27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:45.660394 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d66e1-afcb-41cb-bc4d-a53af80daf27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:45.661793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d66e1-afcb-41cb-bc4d-a53af80daf27 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:45.764885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:45.779015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 636/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 09:48:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 09:48:45.785360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fec99-bbfa-43f0-a5eb-1a1f584243c2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:45.853619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:45.896832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fec99-bbfa-43f0-a5eb-1a1f584243c2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:45.933727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc360d50-b94e-4f5d-adab-d1789828557e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:45.934535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc360d50-b94e-4f5d-adab-d1789828557e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:45.934831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc360d50-b94e-4f5d-adab-d1789828557e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:45.935096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc360d50-b94e-4f5d-adab-d1789828557e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:45.946180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc360d50-b94e-4f5d-adab-d1789828557e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:45.947742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc360d50-b94e-4f5d-adab-d1789828557e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-cc360d50-b94e-4f5d-adab-d1789828557e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 09:48:45.966074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:45.968039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:45.969729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 09:48:45.969729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 09:48:45.969729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 09:48:45.969729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:45.969729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 647/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:45.975211 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d66e1-afcb-41cb-bc4d-a53af80daf27 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:46.087209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d66e1-afcb-41cb-bc4d-a53af80daf27 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:46.123116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf2e61-e23e-41aa-8f65-7f2c8624e942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:46.124184 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf2e61-e23e-41aa-8f65-7f2c8624e942 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:46.124450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf2e61-e23e-41aa-8f65-7f2c8624e942 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:46.124720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf2e61-e23e-41aa-8f65-7f2c8624e942 None admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:46.125448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf2e61-e23e-41aa-8f65-7f2c8624e942 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:46.125751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf2e61-e23e-41aa-8f65-7f2c8624e942 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:46.127150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf2e61-e23e-41aa-8f65-7f2c8624e942 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:46.139789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 637/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 09:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:46.145942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc360d50-b94e-4f5d-adab-d1789828557e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:46.256864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc360d50-b94e-4f5d-adab-d1789828557e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:46.273191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:46.299588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab9c2a-11be-4fa0-b6a8-d1e9255527a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:46.300430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab9c2a-11be-4fa0-b6a8-d1e9255527a1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:46.300708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab9c2a-11be-4fa0-b6a8-d1e9255527a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:46.300963 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab9c2a-11be-4fa0-b6a8-d1e9255527a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:46.309177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab9c2a-11be-4fa0-b6a8-d1e9255527a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:46.310833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab9c2a-11be-4fa0-b6a8-d1e9255527a1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-d6ab9c2a-11be-4fa0-b6a8-d1e9255527a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 09:48:46.326739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:46.328382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:46.330113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 09:48:46.330113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 09:48:46.330113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 09:48:46.330113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 09:48:46.330113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 09:48:46.330113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 09:48:46.330113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 09:48:46.330113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jun 29 09:48:46.330113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 648/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 09:48:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:48:46.337057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf2e61-e23e-41aa-8f65-7f2c8624e942 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:46.396445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:46.450733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf2e61-e23e-41aa-8f65-7f2c8624e942 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:46.488048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7056c1-7817-4eab-a5f0-9d713fbb5f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:46.488778 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7056c1-7817-4eab-a5f0-9d713fbb5f76 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:46.489097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7056c1-7817-4eab-a5f0-9d713fbb5f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:46.489442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7056c1-7817-4eab-a5f0-9d713fbb5f76 None admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:46.490279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7056c1-7817-4eab-a5f0-9d713fbb5f76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:46.490633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7056c1-7817-4eab-a5f0-9d713fbb5f76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:46.492164 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7056c1-7817-4eab-a5f0-9d713fbb5f76 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:46.505965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 638/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 09:48:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:46.512530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab9c2a-11be-4fa0-b6a8-d1e9255527a1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:46.623286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab9c2a-11be-4fa0-b6a8-d1e9255527a1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:46.661409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f6abb-dc41-4137-b2bd-40b9eb2becb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:46.662120 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f6abb-dc41-4137-b2bd-40b9eb2becb5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:46.662436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f6abb-dc41-4137-b2bd-40b9eb2becb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:46.662790 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f6abb-dc41-4137-b2bd-40b9eb2becb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:46.663675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f6abb-dc41-4137-b2bd-40b9eb2becb5 None admin] RBAC: Adding query filter params (role.id=ecc9183a09db4d6b8ec98e96594d33ba, scope.project.id=b4c742f196174c239f6a9c4aba775ee4, user.id=e9bff6c855ea46feaa5526d2c27c9026) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:46.664019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f6abb-dc41-4137-b2bd-40b9eb2becb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:46.665557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f6abb-dc41-4137-b2bd-40b9eb2becb5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:46.672414 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 649/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 09:48:46 2022] GET /identity/v3/role_assignments?user.id=e9bff6c855ea46feaa5526d2c27c9026&scope.project.id=b4c742f196174c239f6a9c4aba775ee4&role.id=ecc9183a09db4d6b8ec98e96594d33ba => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:48.836891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11f715-ff31-4a2d-8c8b-d212bea5ffd1 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:48.837542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11f715-ff31-4a2d-8c8b-d212bea5ffd1 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:48.837631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11f715-ff31-4a2d-8c8b-d212bea5ffd1 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:48.839312 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 639/1288] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:48:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:48:48.846000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04316ee9-ba4f-4cfc-a6f4-29cb16421f41 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:48.846889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04316ee9-ba4f-4cfc-a6f4-29cb16421f41 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:48.846977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04316ee9-ba4f-4cfc-a6f4-29cb16421f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:48.864930 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-04316ee9-ba4f-4cfc-a6f4-29cb16421f41 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:48:48.866122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-04316ee9-ba4f-4cfc-a6f4-29cb16421f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:48:48.898597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:48.939259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 650/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:48:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 09:48:48.962742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11f715-ff31-4a2d-8c8b-d212bea5ffd1 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:49.108453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11f715-ff31-4a2d-8c8b-d212bea5ffd1 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:49.123062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:49.148289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1036c7-2293-445c-ad85-2093190f47e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:49.149114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1036c7-2293-445c-ad85-2093190f47e1 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:49.149390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1036c7-2293-445c-ad85-2093190f47e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:49.149657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1036c7-2293-445c-ad85-2093190f47e1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:49.150330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1036c7-2293-445c-ad85-2093190f47e1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:49.151754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1036c7-2293-445c-ad85-2093190f47e1 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-9c1036c7-2293-445c-ad85-2093190f47e1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:49.164170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:49.165506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:49.166511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 640/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 09:48:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:49.171867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04316ee9-ba4f-4cfc-a6f4-29cb16421f41 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:49.280910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04316ee9-ba4f-4cfc-a6f4-29cb16421f41 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:49.317303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8bcd3-3cb7-4466-8eea-f85795ce4aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:49.317982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8bcd3-3cb7-4466-8eea-f85795ce4aae None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:49.318263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8bcd3-3cb7-4466-8eea-f85795ce4aae None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:49.318533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8bcd3-3cb7-4466-8eea-f85795ce4aae None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:49.319275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8bcd3-3cb7-4466-8eea-f85795ce4aae None admin] RBAC: Adding query filter params (name=placement) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:49.319583 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8bcd3-3cb7-4466-8eea-f85795ce4aae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:49.320968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8bcd3-3cb7-4466-8eea-f85795ce4aae None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:49.326307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 651/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jun 29 09:48:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:49.332621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1036c7-2293-445c-ad85-2093190f47e1 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:49.439468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1036c7-2293-445c-ad85-2093190f47e1 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:49.475948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957acba6-f94a-4024-b137-862d1538feb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:49.476622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957acba6-f94a-4024-b137-862d1538feb2 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:49.476903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957acba6-f94a-4024-b137-862d1538feb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:49.477177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957acba6-f94a-4024-b137-862d1538feb2 None admin] PATH_INFO: `/v3/services` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:49.477895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957acba6-f94a-4024-b137-862d1538feb2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:49.478204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957acba6-f94a-4024-b137-862d1538feb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:49.479617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957acba6-f94a-4024-b137-862d1538feb2 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:49.484674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 641/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jun 29 09:48:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:51.568309 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad62818-1419-4369-8f6f-b94e1abf6ff0 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:51.568896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad62818-1419-4369-8f6f-b94e1abf6ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:51.568896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad62818-1419-4369-8f6f-b94e1abf6ff0 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:51.570343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 652/1293] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:48:51.576339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b195e1-df42-4b45-815a-92dadd84964a None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:51.576915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b195e1-df42-4b45-815a-92dadd84964a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:51.577079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b195e1-df42-4b45-815a-92dadd84964a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:51.596333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b1b195e1-df42-4b45-815a-92dadd84964a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:48:51.597561 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b1b195e1-df42-4b45-815a-92dadd84964a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:48:51.632113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:51.676569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 642/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:48:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:48:51.700976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad62818-1419-4369-8f6f-b94e1abf6ff0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:51.741573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:51.868399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad62818-1419-4369-8f6f-b94e1abf6ff0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:51.935715 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83ea3f-4b5a-456b-a9f6-9a00a1a8af79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:51.936929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83ea3f-4b5a-456b-a9f6-9a00a1a8af79 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:51.937435 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83ea3f-4b5a-456b-a9f6-9a00a1a8af79 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:51.937923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83ea3f-4b5a-456b-a9f6-9a00a1a8af79 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:51.939336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83ea3f-4b5a-456b-a9f6-9a00a1a8af79 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:51.941933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83ea3f-4b5a-456b-a9f6-9a00a1a8af79 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:51.965902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 653/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 09:48:51 2022] POST /identity/v3/services => generated 246 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:48:54.067346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f4324-d1f8-43bf-95b6-5fcfbb186ed1 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:54.068195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f4324-d1f8-43bf-95b6-5fcfbb186ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:54.068352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f4324-d1f8-43bf-95b6-5fcfbb186ed1 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:54.069773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 643/1296] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:48:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:48:54.075668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a0582-4596-4ace-b616-a1233e328c06 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:54.076528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a0582-4596-4ace-b616-a1233e328c06 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:54.076977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a0582-4596-4ace-b616-a1233e328c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:54.095217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-441a0582-4596-4ace-b616-a1233e328c06 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:48:54.096805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-441a0582-4596-4ace-b616-a1233e328c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:48:54.189809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 654/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:48:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:48:54.213481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f4324-d1f8-43bf-95b6-5fcfbb186ed1 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:54.331607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f4324-d1f8-43bf-95b6-5fcfbb186ed1 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:54.373808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71b58b-163c-4add-ba30-d1a65274be73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:54.374716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71b58b-163c-4add-ba30-d1a65274be73 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:54.375004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71b58b-163c-4add-ba30-d1a65274be73 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:54.375281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71b58b-163c-4add-ba30-d1a65274be73 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:54.375950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71b58b-163c-4add-ba30-d1a65274be73 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:54.377378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71b58b-163c-4add-ba30-d1a65274be73 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-2d71b58b-163c-4add-ba30-d1a65274be73 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:54.390822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:54.392684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:54.394020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 644/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 09:48:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 09:48:54.399236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a0582-4596-4ace-b616-a1233e328c06 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:54.426016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:54.550611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a0582-4596-4ace-b616-a1233e328c06 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:54.593768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadff48-5863-4403-bc81-8b6456cd6a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:54.594417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadff48-5863-4403-bc81-8b6456cd6a99 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:54.594793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadff48-5863-4403-bc81-8b6456cd6a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:54.595121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadff48-5863-4403-bc81-8b6456cd6a99 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:54.595899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eadff48-5863-4403-bc81-8b6456cd6a99 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:54.596607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eadff48-5863-4403-bc81-8b6456cd6a99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:54.598088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eadff48-5863-4403-bc81-8b6456cd6a99 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:54.604143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 655/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jun 29 09:48:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:54.611032 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71b58b-163c-4add-ba30-d1a65274be73 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:54.644716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:54.722176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71b58b-163c-4add-ba30-d1a65274be73 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:54.760425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2abc4-bcaa-48c7-8063-f77b42529128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:54.761088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2abc4-bcaa-48c7-8063-f77b42529128 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:54.761367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2abc4-bcaa-48c7-8063-f77b42529128 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:54.761641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2abc4-bcaa-48c7-8063-f77b42529128 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:54.762444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2abc4-bcaa-48c7-8063-f77b42529128 None admin] RBAC: Adding query filter params (interface=public, service_id=e99f962ac4d24b7db3a1392bd1232581, region_id=RegionOne) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:54.762786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2abc4-bcaa-48c7-8063-f77b42529128 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:54.764221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2abc4-bcaa-48c7-8063-f77b42529128 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:54.770510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 645/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 09:48:54 2022] GET /identity/v3/endpoints?service_id=e99f962ac4d24b7db3a1392bd1232581&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:54.779099 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadff48-5863-4403-bc81-8b6456cd6a99 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:54.871686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:54.892471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadff48-5863-4403-bc81-8b6456cd6a99 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:54.930320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5925dca-98bc-4b43-a104-bb027f713e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:54.931032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5925dca-98bc-4b43-a104-bb027f713e3c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:54.931306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5925dca-98bc-4b43-a104-bb027f713e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:54.931573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5925dca-98bc-4b43-a104-bb027f713e3c None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:54.932255 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5925dca-98bc-4b43-a104-bb027f713e3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:54.933644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5925dca-98bc-4b43-a104-bb027f713e3c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:54.941111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 656/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 09:48:54 2022] GET /identity/v3/services => generated 2191 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:48:56.999674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029be01-0fd5-418e-a183-9e68ce3f0399 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:57.000532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029be01-0fd5-418e-a183-9e68ce3f0399 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:57.000532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029be01-0fd5-418e-a183-9e68ce3f0399 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:57.001650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 646/1302] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:48:57.008564 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f6e4f-e627-482f-8f0b-f741c67728df None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:57.009089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f6e4f-e627-482f-8f0b-f741c67728df None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:57.009542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f6e4f-e627-482f-8f0b-f741c67728df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:57.032003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b23f6e4f-e627-482f-8f0b-f741c67728df None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:48:57.033217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b23f6e4f-e627-482f-8f0b-f741c67728df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:48:57.107122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 657/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:48:57 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 09:48:57.131008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029be01-0fd5-418e-a183-9e68ce3f0399 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:57.274648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:57.288421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029be01-0fd5-418e-a183-9e68ce3f0399 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:57.356736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163ab32-c143-499f-835b-36223c8ce181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:57.358474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163ab32-c143-499f-835b-36223c8ce181 None admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:57.359019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163ab32-c143-499f-835b-36223c8ce181 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:57.359517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163ab32-c143-499f-835b-36223c8ce181 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:57.360776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163ab32-c143-499f-835b-36223c8ce181 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:57.363490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163ab32-c143-499f-835b-36223c8ce181 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-1163ab32-c143-499f-835b-36223c8ce181 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 09:48:57.387813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 09:48:57.389624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 09:48:57.390463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 647/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 09:48:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 09:48:57.399253 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f6e4f-e627-482f-8f0b-f741c67728df None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:57.480505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:48:57.505983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f6e4f-e627-482f-8f0b-f741c67728df None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:57.543821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19354d-1d55-4241-bb7f-de27b6d3bae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:57.544391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19354d-1d55-4241-bb7f-de27b6d3bae2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:57.544697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19354d-1d55-4241-bb7f-de27b6d3bae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:57.544958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19354d-1d55-4241-bb7f-de27b6d3bae2 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:57.545686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19354d-1d55-4241-bb7f-de27b6d3bae2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:48:57.545989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19354d-1d55-4241-bb7f-de27b6d3bae2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:57.547438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19354d-1d55-4241-bb7f-de27b6d3bae2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:57.553346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 658/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jun 29 09:48:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:48:57.559585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163ab32-c143-499f-835b-36223c8ce181 None admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:48:57.671233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163ab32-c143-499f-835b-36223c8ce181 None admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:48:57.708305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fdbb3-b695-4625-8ebd-80002510fa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:48:57.708968 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fdbb3-b695-4625-8ebd-80002510fa06 None admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:48:57.709245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fdbb3-b695-4625-8ebd-80002510fa06 None admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:48:57.709514 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fdbb3-b695-4625-8ebd-80002510fa06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:48:57.710269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fdbb3-b695-4625-8ebd-80002510fa06 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:48:57.711663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fdbb3-b695-4625-8ebd-80002510fa06 None admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:48:57.733256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 648/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 09:48:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:49:07.550620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=30 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:49:07.728038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:08.225413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5716c-1641-4c36-8be5-5c7ab3a56fd6 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:08.226003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5716c-1641-4c36-8be5-5c7ab3a56fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:08.226003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5716c-1641-4c36-8be5-5c7ab3a56fd6 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:08.233026 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c1959-6b12-46c1-a8cf-747f85ecb8bf None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:08.233530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c1959-6b12-46c1-a8cf-747f85ecb8bf None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:08.233621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c1959-6b12-46c1-a8cf-747f85ecb8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:08.237174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 659/1307] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:08 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:08.332000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-cc7c1959-6b12-46c1-a8cf-747f85ecb8bf None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:08.333231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cc7c1959-6b12-46c1-a8cf-747f85ecb8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:08.487164 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 649/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:08.553501 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857995c-0102-4255-9819-47029cd2c6cc None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:08.554489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857995c-0102-4255-9819-47029cd2c6cc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:08.555335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857995c-0102-4255-9819-47029cd2c6cc None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:08.567596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 660/1309] 10.222.0.52 () {54 vars in 964 bytes} [Wed Jun 29 09:50:08 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:08.585110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc8cee-d817-4e44-9f3d-27d13bf54364 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:08.585396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc8cee-d817-4e44-9f3d-27d13bf54364 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:08.585684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc8cee-d817-4e44-9f3d-27d13bf54364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:08.670909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-c3bc8cee-d817-4e44-9f3d-27d13bf54364 None None] MFA Rules not processed for user `4c97b321bf854177bd2f310cf2958959`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:08.672423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c3bc8cee-d817-4e44-9f3d-27d13bf54364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:08.704142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:08.770418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 650/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Jun 29 09:50:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:08.779775 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857995c-0102-4255-9819-47029cd2c6cc None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:08.781552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8857995c-0102-4255-9819-47029cd2c6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:08.903014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857995c-0102-4255-9819-47029cd2c6cc None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:09.003256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee576d3-7ada-4201-8e68-f87036e49d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:09.003884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee576d3-7ada-4201-8e68-f87036e49d28 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:09.004163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee576d3-7ada-4201-8e68-f87036e49d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:09.004517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee576d3-7ada-4201-8e68-f87036e49d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:09.006271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dee576d3-7ada-4201-8e68-f87036e49d28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:09.078949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:09.141436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee576d3-7ada-4201-8e68-f87036e49d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:09.143062 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee576d3-7ada-4201-8e68-f87036e49d28 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:09.165525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 661/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:50:11.544260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cf1b4-c242-4a64-a8da-0b7b4d798a33 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:11.545321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cf1b4-c242-4a64-a8da-0b7b4d798a33 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:11.545321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cf1b4-c242-4a64-a8da-0b7b4d798a33 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:11.546092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 651/1312] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:11.551953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f25d5-2bed-4c83-8849-099947f9cade None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:11.552358 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f25d5-2bed-4c83-8849-099947f9cade None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:11.552496 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f25d5-2bed-4c83-8849-099947f9cade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:11.626526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c47f25d5-2bed-4c83-8849-099947f9cade None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:11.627757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c47f25d5-2bed-4c83-8849-099947f9cade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:11.738219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 662/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:11.788725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb3d4-4b66-421d-899c-be556c84e24c None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:11.789021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb3d4-4b66-421d-899c-be556c84e24c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:11.789296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb3d4-4b66-421d-899c-be556c84e24c None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:11.790737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 652/1314] 10.222.0.52 () {54 vars in 964 bytes} [Wed Jun 29 09:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:11.797802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9ac5a-aa12-4e62-9562-406aa549ea19 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:11.798088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9ac5a-aa12-4e62-9562-406aa549ea19 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:11.798356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9ac5a-aa12-4e62-9562-406aa549ea19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:11.809152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:11.878723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dae9ac5a-aa12-4e62-9562-406aa549ea19 None None] MFA Rules not processed for user `4c97b321bf854177bd2f310cf2958959`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:11.879942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dae9ac5a-aa12-4e62-9562-406aa549ea19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:11.954422 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 663/1315] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Jun 29 09:50:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:11.963773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eb3d4-4b66-421d-899c-be556c84e24c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:11.965311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-993eb3d4-4b66-421d-899c-be556c84e24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:12.073259 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eb3d4-4b66-421d-899c-be556c84e24c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:12.148308 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:12.151358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a087db3-241f-4d81-beda-897368b92274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:12.152370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a087db3-241f-4d81-beda-897368b92274 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:12.152952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a087db3-241f-4d81-beda-897368b92274 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:12.153298 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a087db3-241f-4d81-beda-897368b92274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:12.155520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-0a087db3-241f-4d81-beda-897368b92274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:12.279144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a087db3-241f-4d81-beda-897368b92274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:12.280605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a087db3-241f-4d81-beda-897368b92274 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:12.300450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 653/1316] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:50:14.880548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a9211-bc89-4c56-b144-b2f417fd1f84 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:14.881121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a9211-bc89-4c56-b144-b2f417fd1f84 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:14.881121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a9211-bc89-4c56-b144-b2f417fd1f84 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:14.882512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 664/1317] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:14.889232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307cd3c-da7e-4f95-baf2-6d303e25692b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:14.889707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307cd3c-da7e-4f95-baf2-6d303e25692b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:14.889799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307cd3c-da7e-4f95-baf2-6d303e25692b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:14.958418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-d307cd3c-da7e-4f95-baf2-6d303e25692b None None] MFA Rules not processed for user `b91b2cc826954cfd869846d1c5f8c6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:14.959702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d307cd3c-da7e-4f95-baf2-6d303e25692b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:15.014893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:15.069870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 654/1318] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:15.110805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a9211-bc89-4c56-b144-b2f417fd1f84 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:15.112396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d52a9211-bc89-4c56-b144-b2f417fd1f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:15.221957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a9211-bc89-4c56-b144-b2f417fd1f84 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:15.224794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:15.299705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3681c59-d9ce-49ff-b50a-990816e32c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:15.300346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3681c59-d9ce-49ff-b50a-990816e32c6b service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:15.300624 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3681c59-d9ce-49ff-b50a-990816e32c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:15.300902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3681c59-d9ce-49ff-b50a-990816e32c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:15.302549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e3681c59-d9ce-49ff-b50a-990816e32c6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:15.417674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3681c59-d9ce-49ff-b50a-990816e32c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:15.419439 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3681c59-d9ce-49ff-b50a-990816e32c6b service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:15.440514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 665/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:15 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:50:18.237105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e933a7-d113-45a0-b55c-80de8ae6758a None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:18.237765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e933a7-d113-45a0-b55c-80de8ae6758a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:18.237765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e933a7-d113-45a0-b55c-80de8ae6758a None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:18.239073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 655/1320] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:18.244420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5434ef8-4ee1-43dd-80bd-a4852a7404cc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:18.245134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5434ef8-4ee1-43dd-80bd-a4852a7404cc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:18.245369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5434ef8-4ee1-43dd-80bd-a4852a7404cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:18.287666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:18.315635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e5434ef8-4ee1-43dd-80bd-a4852a7404cc None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:18.316846 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e5434ef8-4ee1-43dd-80bd-a4852a7404cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:18.418876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 666/1321] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:18.457585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e933a7-d113-45a0-b55c-80de8ae6758a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:18.459153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e3e933a7-d113-45a0-b55c-80de8ae6758a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:18.572903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e933a7-d113-45a0-b55c-80de8ae6758a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:18.649405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e08a6-30bf-44d7-a70b-bf6f014bacf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:18.650056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e08a6-30bf-44d7-a70b-bf6f014bacf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:18.650327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e08a6-30bf-44d7-a70b-bf6f014bacf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:18.650617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e08a6-30bf-44d7-a70b-bf6f014bacf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:18.652262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a81e08a6-30bf-44d7-a70b-bf6f014bacf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:18.660357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:18.766811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e08a6-30bf-44d7-a70b-bf6f014bacf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:18.768273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e08a6-30bf-44d7-a70b-bf6f014bacf8 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:18.787099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 656/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:50:21.550516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e3f11-8607-4096-ab06-1d8c4f3b70c0 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:21.551232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e3f11-8607-4096-ab06-1d8c4f3b70c0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:21.551232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e3f11-8607-4096-ab06-1d8c4f3b70c0 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:21.552487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 667/1323] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:21.557928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cca9e7-9848-41c5-ae51-c2fcfafad01b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:21.558379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cca9e7-9848-41c5-ae51-c2fcfafad01b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:21.558658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cca9e7-9848-41c5-ae51-c2fcfafad01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:21.627103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-76cca9e7-9848-41c5-ae51-c2fcfafad01b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:21.628287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-76cca9e7-9848-41c5-ae51-c2fcfafad01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:21.692316 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:21.737423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 657/1324] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:21.773485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e3f11-8607-4096-ab06-1d8c4f3b70c0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:21.775075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a03e3f11-8607-4096-ab06-1d8c4f3b70c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:21.893307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e3f11-8607-4096-ab06-1d8c4f3b70c0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:21.908205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:21.976925 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac6cb4-0b65-4004-937a-13781d2a96b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:21.977587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac6cb4-0b65-4004-937a-13781d2a96b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:21.977863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac6cb4-0b65-4004-937a-13781d2a96b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:21.978134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac6cb4-0b65-4004-937a-13781d2a96b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:21.979890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-66ac6cb4-0b65-4004-937a-13781d2a96b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:22.097133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac6cb4-0b65-4004-937a-13781d2a96b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:22.098918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac6cb4-0b65-4004-937a-13781d2a96b0 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:22.119064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 668/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:50:24.298361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9da18-a805-45fe-9dbe-520bafc9f95a None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:24.298940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9da18-a805-45fe-9dbe-520bafc9f95a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:24.298940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9da18-a805-45fe-9dbe-520bafc9f95a None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:24.300245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 658/1326] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:24.306010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031ee7c-ca34-412c-b92e-8ea90089168f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:24.306498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031ee7c-ca34-412c-b92e-8ea90089168f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:24.306498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031ee7c-ca34-412c-b92e-8ea90089168f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:24.358692 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:24.377800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5031ee7c-ca34-412c-b92e-8ea90089168f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:24.379040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5031ee7c-ca34-412c-b92e-8ea90089168f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:24.479921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 669/1327] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:24.516798 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9da18-a805-45fe-9dbe-520bafc9f95a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:24.518484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-54d9da18-a805-45fe-9dbe-520bafc9f95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:24.632473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9da18-a805-45fe-9dbe-520bafc9f95a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:24.714255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85096a-873b-4033-8020-9ef856a5ec9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:24.715033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85096a-873b-4033-8020-9ef856a5ec9b service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:24.715314 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85096a-873b-4033-8020-9ef856a5ec9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:24.715814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85096a-873b-4033-8020-9ef856a5ec9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:24.717608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4e85096a-873b-4033-8020-9ef856a5ec9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:24.736167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:24.837706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85096a-873b-4033-8020-9ef856a5ec9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:24.839194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85096a-873b-4033-8020-9ef856a5ec9b service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:24.859445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 659/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:50:26.987912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accff95-100b-4df5-9533-747bc3493349 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:26.988491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accff95-100b-4df5-9533-747bc3493349 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:26.988491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accff95-100b-4df5-9533-747bc3493349 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:26.989816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 670/1329] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:26.996031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee063d92-6229-415c-a9e1-063bc07b52ef None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:26.996542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee063d92-6229-415c-a9e1-063bc07b52ef None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:26.996542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee063d92-6229-415c-a9e1-063bc07b52ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:27.070095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-ee063d92-6229-415c-a9e1-063bc07b52ef None None] MFA Rules not processed for user `b91b2cc826954cfd869846d1c5f8c6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:27.071364 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ee063d92-6229-415c-a9e1-063bc07b52ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:27.144353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:27.181751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 660/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:27.219902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accff95-100b-4df5-9533-747bc3493349 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:27.221877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3accff95-100b-4df5-9533-747bc3493349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:27.333526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accff95-100b-4df5-9533-747bc3493349 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:27.354437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:27.411240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1411eba-a50e-4953-bbe2-6bfc03a2ebf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:27.411900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1411eba-a50e-4953-bbe2-6bfc03a2ebf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:27.412180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1411eba-a50e-4953-bbe2-6bfc03a2ebf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:27.412453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1411eba-a50e-4953-bbe2-6bfc03a2ebf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:27.414131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b1411eba-a50e-4953-bbe2-6bfc03a2ebf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:27.548039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1411eba-a50e-4953-bbe2-6bfc03a2ebf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:27.549433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1411eba-a50e-4953-bbe2-6bfc03a2ebf9 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:27.569743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 671/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:50:29.953332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749df1b3-ec09-4df0-ac8c-8bad72c83e82 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:29.953928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749df1b3-ec09-4df0-ac8c-8bad72c83e82 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:29.953928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749df1b3-ec09-4df0-ac8c-8bad72c83e82 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:29.955508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 661/1332] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:29.961013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c40ce-f6b0-47d3-bec1-eeda5af4d96d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:29.961524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c40ce-f6b0-47d3-bec1-eeda5af4d96d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:29.961626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c40ce-f6b0-47d3-bec1-eeda5af4d96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:30.032089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:30.035466 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4e9c40ce-f6b0-47d3-bec1-eeda5af4d96d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:30.037392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4e9c40ce-f6b0-47d3-bec1-eeda5af4d96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:30.138904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 672/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:30.174708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749df1b3-ec09-4df0-ac8c-8bad72c83e82 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:30.176262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-749df1b3-ec09-4df0-ac8c-8bad72c83e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:30.283770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749df1b3-ec09-4df0-ac8c-8bad72c83e82 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:30.361060 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f045eab-e6f8-4c54-b575-a69a7e363953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:30.361715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045eab-e6f8-4c54-b575-a69a7e363953 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:30.361996 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045eab-e6f8-4c54-b575-a69a7e363953 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:30.362265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045eab-e6f8-4c54-b575-a69a7e363953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:30.363956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2f045eab-e6f8-4c54-b575-a69a7e363953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:30.392795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:30.482582 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f045eab-e6f8-4c54-b575-a69a7e363953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:30.484034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f045eab-e6f8-4c54-b575-a69a7e363953 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:30.503481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 662/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:50:32.588391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb47491-88df-4409-893c-57decb1c2883 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:32.589030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb47491-88df-4409-893c-57decb1c2883 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:32.589030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb47491-88df-4409-893c-57decb1c2883 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:32.590368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 673/1335] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:32.596500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccd76b-ff60-4499-9653-8235a226da63 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:32.597012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccd76b-ff60-4499-9653-8235a226da63 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:32.597012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccd76b-ff60-4499-9653-8235a226da63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:32.666820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-0fccd76b-ff60-4499-9653-8235a226da63 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:32.668044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-0fccd76b-ff60-4499-9653-8235a226da63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:32.745135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:32.770760 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 663/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:32.811730 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb47491-88df-4409-893c-57decb1c2883 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:32.813295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ccb47491-88df-4409-893c-57decb1c2883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:32.939281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb47491-88df-4409-893c-57decb1c2883 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:32.972478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:33.029203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623515a-3c2d-4199-84a8-984e52531d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:33.029871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623515a-3c2d-4199-84a8-984e52531d2d service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:33.030154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623515a-3c2d-4199-84a8-984e52531d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:33.030432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623515a-3c2d-4199-84a8-984e52531d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:33.032201 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3623515a-3c2d-4199-84a8-984e52531d2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:33.174923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623515a-3c2d-4199-84a8-984e52531d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:33.176347 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623515a-3c2d-4199-84a8-984e52531d2d service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:33.196385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 674/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:50:35.605857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fca93-ec3c-4061-a8b7-4a62ece96cbf None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:35.606690 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fca93-ec3c-4061-a8b7-4a62ece96cbf None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:35.606690 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fca93-ec3c-4061-a8b7-4a62ece96cbf None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:35.607993 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 664/1338] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:35.613538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a0455-e4ab-42e9-9f37-188a362859a2 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:35.614032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a0455-e4ab-42e9-9f37-188a362859a2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:35.614128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a0455-e4ab-42e9-9f37-188a362859a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:35.682521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-727a0455-e4ab-42e9-9f37-188a362859a2 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:35.683804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-727a0455-e4ab-42e9-9f37-188a362859a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:35.693051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:35.790225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 675/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:35.831511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fca93-ec3c-4061-a8b7-4a62ece96cbf None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:35.833145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4c2fca93-ec3c-4061-a8b7-4a62ece96cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:35.945149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fca93-ec3c-4061-a8b7-4a62ece96cbf None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:36.025593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6e956-d889-4e88-8413-86b98d7d7f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:36.026272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e956-d889-4e88-8413-86b98d7d7f2f service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:36.026553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e956-d889-4e88-8413-86b98d7d7f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:36.026861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e956-d889-4e88-8413-86b98d7d7f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:36.028847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-97f6e956-d889-4e88-8413-86b98d7d7f2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:36.069165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:36.164454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e956-d889-4e88-8413-86b98d7d7f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:36.166232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e956-d889-4e88-8413-86b98d7d7f2f service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:36.187422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 665/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:50:40.293820 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e96b1-7dbc-4afa-8734-166c3191bd3e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:40.294364 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e96b1-7dbc-4afa-8734-166c3191bd3e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:40.294364 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e96b1-7dbc-4afa-8734-166c3191bd3e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:40.295852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 676/1341] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:40.301146 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb1f95-b8cb-446a-bd07-2fefa7562a4b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:40.301583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb1f95-b8cb-446a-bd07-2fefa7562a4b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:40.301695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb1f95-b8cb-446a-bd07-2fefa7562a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:40.376904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2bdb1f95-b8cb-446a-bd07-2fefa7562a4b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:40.378122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2bdb1f95-b8cb-446a-bd07-2fefa7562a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:40.475716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:40.489767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 666/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:40.528533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e96b1-7dbc-4afa-8734-166c3191bd3e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:40.531557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-786e96b1-7dbc-4afa-8734-166c3191bd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:40.636420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e96b1-7dbc-4afa-8734-166c3191bd3e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:40.640345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2c08a-d23f-4880-845d-a846ac27ac78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:40.641032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2c08a-d23f-4880-845d-a846ac27ac78 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:40.641276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2c08a-d23f-4880-845d-a846ac27ac78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:40.641551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2c08a-d23f-4880-845d-a846ac27ac78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:40.651072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2c08a-d23f-4880-845d-a846ac27ac78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:40.652529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2c08a-d23f-4880-845d-a846ac27ac78 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:40.662482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 677/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:50:43.523331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daa320-4d9e-4a03-9b15-1dd007a48164 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:43.523987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daa320-4d9e-4a03-9b15-1dd007a48164 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:43.523987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daa320-4d9e-4a03-9b15-1dd007a48164 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:43.525321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 667/1344] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:43.532124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1da88-11df-475f-9045-640e75d70845 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:43.532759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1da88-11df-475f-9045-640e75d70845 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:43.532759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1da88-11df-475f-9045-640e75d70845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:43.563597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-efe1da88-11df-475f-9045-640e75d70845 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:43.564839 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-efe1da88-11df-475f-9045-640e75d70845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:43.598450 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:43.652066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 678/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:43.690212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2daa320-4d9e-4a03-9b15-1dd007a48164 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:43.692058 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e2daa320-4d9e-4a03-9b15-1dd007a48164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:43.707682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2daa320-4d9e-4a03-9b15-1dd007a48164 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:43.711880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b5ce1-8644-4e5e-ad18-6070c227001e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:43.712558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b5ce1-8644-4e5e-ad18-6070c227001e service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:43.712838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b5ce1-8644-4e5e-ad18-6070c227001e service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:43.713087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b5ce1-8644-4e5e-ad18-6070c227001e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:43.725275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b5ce1-8644-4e5e-ad18-6070c227001e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:43.726751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b5ce1-8644-4e5e-ad18-6070c227001e service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:43.737255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 668/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:50:47.861535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382523cb-4571-4886-a1d6-f334b812b806 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:47.862170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382523cb-4571-4886-a1d6-f334b812b806 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:47.862170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382523cb-4571-4886-a1d6-f334b812b806 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:47.863507 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 679/1347] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:47.869002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0bf1-e466-4423-b219-a3785ba494bb None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:47.869542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0bf1-e466-4423-b219-a3785ba494bb None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:47.869542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0bf1-e466-4423-b219-a3785ba494bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:47.887751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-9cde0bf1-e466-4423-b219-a3785ba494bb None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:47.889118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9cde0bf1-e466-4423-b219-a3785ba494bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:47.897228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 669/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:47.935328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382523cb-4571-4886-a1d6-f334b812b806 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:50:47.947505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382523cb-4571-4886-a1d6-f334b812b806 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:50:47.951954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8f866-cb98-4f55-8a06-89b2437b5b72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:50:47.952706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8f866-cb98-4f55-8a06-89b2437b5b72 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:47.953057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8f866-cb98-4f55-8a06-89b2437b5b72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:47.953438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8f866-cb98-4f55-8a06-89b2437b5b72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:47.963697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8f866-cb98-4f55-8a06-89b2437b5b72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:50:47.965530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8f866-cb98-4f55-8a06-89b2437b5b72 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:50:47.977170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 680/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:50:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:50:53.211115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a32daf-8554-467b-8241-9362c45d5d9d None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:53.211966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a32daf-8554-467b-8241-9362c45d5d9d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:53.212413 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a32daf-8554-467b-8241-9362c45d5d9d None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:53.214072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 670/1350] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 09:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:53.223864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f844b-06d2-4e73-91fa-e7cd497211ee None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:53.224808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f844b-06d2-4e73-91fa-e7cd497211ee None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:53.225753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f844b-06d2-4e73-91fa-e7cd497211ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:53.309050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5c5f844b-06d2-4e73-91fa-e7cd497211ee None None] MFA Rules not processed for user `e9bff6c855ea46feaa5526d2c27c9026`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:53.310268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5c5f844b-06d2-4e73-91fa-e7cd497211ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:53.415478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 681/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 09:50:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:57.862170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06590a-e003-4a07-a860-c1d5fd34c20c None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:57.862822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06590a-e003-4a07-a860-c1d5fd34c20c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:57.862822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06590a-e003-4a07-a860-c1d5fd34c20c None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:57.864759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 671/1352] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 09:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:57.869920 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a9735-e4f0-4a37-9b76-4f379c7620fc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:57.870524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a9735-e4f0-4a37-9b76-4f379c7620fc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:57.870926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a9735-e4f0-4a37-9b76-4f379c7620fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:57.882374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=13 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:50:57.891098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-413a9735-e4f0-4a37-9b76-4f379c7620fc None None] MFA Rules not processed for user `e9bff6c855ea46feaa5526d2c27c9026`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:57.892317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-413a9735-e4f0-4a37-9b76-4f379c7620fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:57.901630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 682/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 09:50:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:50:59.320127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a9e0-6b06-4704-bdef-ed1d527c51fe None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:59.320650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a9e0-6b06-4704-bdef-ed1d527c51fe None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:59.320650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755a9e0-6b06-4704-bdef-ed1d527c51fe None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:59.321934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 672/1354] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 09:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:50:59.327041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5809a0b-c0e6-447c-8d8e-651c73ef9e12 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:50:59.327495 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5809a0b-c0e6-447c-8d8e-651c73ef9e12 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:50:59.327687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5809a0b-c0e6-447c-8d8e-651c73ef9e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:50:59.346923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d5809a0b-c0e6-447c-8d8e-651c73ef9e12 None None] MFA Rules not processed for user `e9bff6c855ea46feaa5526d2c27c9026`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:50:59.348131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d5809a0b-c0e6-447c-8d8e-651c73ef9e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:50:59.357521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 683/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 09:50:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:02.885882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66b4a0-aa15-42c9-9a92-5650c275519e None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:02.886513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66b4a0-aa15-42c9-9a92-5650c275519e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:02.886733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66b4a0-aa15-42c9-9a92-5650c275519e None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:02.889178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 673/1356] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:02.897827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7306ad-6283-4ec2-9ed6-8e65e1d01775 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:02.898393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7306ad-6283-4ec2-9ed6-8e65e1d01775 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:02.898637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7306ad-6283-4ec2-9ed6-8e65e1d01775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:02.927104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0d7306ad-6283-4ec2-9ed6-8e65e1d01775 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:02.928788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0d7306ad-6283-4ec2-9ed6-8e65e1d01775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:02.941366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 684/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:03.001077 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9afc-ca8b-48a3-b3ef-6e3090b9c300 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:03.004149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9afc-ca8b-48a3-b3ef-6e3090b9c300 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:03.006633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9afc-ca8b-48a3-b3ef-6e3090b9c300 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:03.016618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 674/1358] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jun 29 09:51:02 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:03.026633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5adbb-1f52-4a9c-8cef-98c52ca9b0c2 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:03.027502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5adbb-1f52-4a9c-8cef-98c52ca9b0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:03.028260 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5adbb-1f52-4a9c-8cef-98c52ca9b0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:03.079456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:51:03.089056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6ca5adbb-1f52-4a9c-8cef-98c52ca9b0c2 None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:03.090485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6ca5adbb-1f52-4a9c-8cef-98c52ca9b0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:03.183874 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 685/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jun 29 09:51:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:03.193627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea9afc-ca8b-48a3-b3ef-6e3090b9c300 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:03.208902 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea9afc-ca8b-48a3-b3ef-6e3090b9c300 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:03.214694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed10ab3-2af1-40a0-80bf-8ffbf5adead9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:03.215495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed10ab3-2af1-40a0-80bf-8ffbf5adead9 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:03.215907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed10ab3-2af1-40a0-80bf-8ffbf5adead9 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:03.216267 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed10ab3-2af1-40a0-80bf-8ffbf5adead9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:03.225653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed10ab3-2af1-40a0-80bf-8ffbf5adead9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:03.227419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed10ab3-2af1-40a0-80bf-8ffbf5adead9 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:03.238811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 675/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:51:05.527381 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539d120-8de2-4195-81a3-c90f3e750d28 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:05.529142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539d120-8de2-4195-81a3-c90f3e750d28 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:05.529563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539d120-8de2-4195-81a3-c90f3e750d28 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:05.531175 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 686/1361] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:05.538982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a8021-96b0-4802-b072-2e09404a508a None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:05.539547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a8021-96b0-4802-b072-2e09404a508a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:05.539547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a8021-96b0-4802-b072-2e09404a508a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:05.563575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6d9a8021-96b0-4802-b072-2e09404a508a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:05.564944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6d9a8021-96b0-4802-b072-2e09404a508a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:05.576136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 676/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:51:05.630821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539d120-8de2-4195-81a3-c90f3e750d28 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:05.652232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539d120-8de2-4195-81a3-c90f3e750d28 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:05.659557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35894777-0bf4-4f48-8d00-e43035c67966 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:05.660360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35894777-0bf4-4f48-8d00-e43035c67966 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:05.660763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35894777-0bf4-4f48-8d00-e43035c67966 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:05.661118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35894777-0bf4-4f48-8d00-e43035c67966 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:05.672365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35894777-0bf4-4f48-8d00-e43035c67966 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:05.674233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35894777-0bf4-4f48-8d00-e43035c67966 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:05.688599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 687/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:08.320282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980947b9-9cc7-45df-bef4-4d881fed2025 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:08.320867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980947b9-9cc7-45df-bef4-4d881fed2025 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:08.320867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980947b9-9cc7-45df-bef4-4d881fed2025 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:08.322336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 677/1364] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:08.329327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9aacb-87e8-431a-bf31-21bb52662779 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:08.329770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9aacb-87e8-431a-bf31-21bb52662779 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:08.330067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9aacb-87e8-431a-bf31-21bb52662779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:08.353298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-53f9aacb-87e8-431a-bf31-21bb52662779 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:08.353553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-53f9aacb-87e8-431a-bf31-21bb52662779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:08.366094 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 688/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:08.419713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605cbf9-5c37-4978-9d1e-0aec1dee2eed None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:08.420031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605cbf9-5c37-4978-9d1e-0aec1dee2eed None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:08.420280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605cbf9-5c37-4978-9d1e-0aec1dee2eed None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:08.421685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 678/1366] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jun 29 09:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:08.427247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bdbac-71a9-4e5f-b660-09254f687d87 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:08.427753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bdbac-71a9-4e5f-b660-09254f687d87 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:08.428171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bdbac-71a9-4e5f-b660-09254f687d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:08.454740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-674bdbac-71a9-4e5f-b660-09254f687d87 None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:08.455966 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-674bdbac-71a9-4e5f-b660-09254f687d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:08.471454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 689/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jun 29 09:51:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:08.483932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605cbf9-5c37-4978-9d1e-0aec1dee2eed None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:08.500755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605cbf9-5c37-4978-9d1e-0aec1dee2eed None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:08.506727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0a768-6697-4b03-96e2-f0804474dccf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:08.507417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0a768-6697-4b03-96e2-f0804474dccf service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:08.507723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0a768-6697-4b03-96e2-f0804474dccf service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:08.508056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0a768-6697-4b03-96e2-f0804474dccf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:08.518365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0a768-6697-4b03-96e2-f0804474dccf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:08.520017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0a768-6697-4b03-96e2-f0804474dccf service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:08.530504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 679/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:51:10.680863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4337a-c6c1-453a-8f75-3ed8b8c34a51 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:10.681517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4337a-c6c1-453a-8f75-3ed8b8c34a51 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:10.681517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4337a-c6c1-453a-8f75-3ed8b8c34a51 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:10.682855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 690/1369] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:10.688813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bab03-6b8c-43dc-ad57-7f0cde050e0e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:10.689410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bab03-6b8c-43dc-ad57-7f0cde050e0e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:10.689410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bab03-6b8c-43dc-ad57-7f0cde050e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:10.708769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-220bab03-6b8c-43dc-ad57-7f0cde050e0e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:10.710017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-220bab03-6b8c-43dc-ad57-7f0cde050e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:10.718607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 680/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:10.767262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4337a-c6c1-453a-8f75-3ed8b8c34a51 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:10.783558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4337a-c6c1-453a-8f75-3ed8b8c34a51 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:10.789633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df1723-bfe7-4360-ba02-0d9b4836ddc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:10.790468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df1723-bfe7-4360-ba02-0d9b4836ddc1 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:10.790890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df1723-bfe7-4360-ba02-0d9b4836ddc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:10.791259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df1723-bfe7-4360-ba02-0d9b4836ddc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:10.802263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df1723-bfe7-4360-ba02-0d9b4836ddc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:10.804117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df1723-bfe7-4360-ba02-0d9b4836ddc1 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:10.815745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 691/1371] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:11.618029 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b158a-a286-489d-9d01-f71f317286b5 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:11.618313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b158a-a286-489d-9d01-f71f317286b5 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:11.618809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b158a-a286-489d-9d01-f71f317286b5 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:11.620230 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 681/1372] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:51:11.627418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84ae92-f1c2-48bb-b9c6-b72c1d9fca5d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:11.627958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84ae92-f1c2-48bb-b9c6-b72c1d9fca5d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:11.628426 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84ae92-f1c2-48bb-b9c6-b72c1d9fca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:11.649117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1a84ae92-f1c2-48bb-b9c6-b72c1d9fca5d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:11.650764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1a84ae92-f1c2-48bb-b9c6-b72c1d9fca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:11.659653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 692/1373] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:12.022247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af27fe5-7fd6-498d-994d-033465b3b8dd None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:12.022247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af27fe5-7fd6-498d-994d-033465b3b8dd None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:12.022247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af27fe5-7fd6-498d-994d-033465b3b8dd None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:12.024826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 682/1374] 10.222.0.52 () {54 vars in 963 bytes} [Wed Jun 29 09:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:12.029774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ce992-f0db-4ccb-9333-d6d2f7e68ca7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:12.029774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ce992-f0db-4ccb-9333-d6d2f7e68ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:12.030441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ce992-f0db-4ccb-9333-d6d2f7e68ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:12.098720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a44ce992-f0db-4ccb-9333-d6d2f7e68ca7 None None] MFA Rules not processed for user `34813be7e8fd4fcfb5907bee6a465d16`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:12.100096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a44ce992-f0db-4ccb-9333-d6d2f7e68ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:12.190959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 693/1375] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Jun 29 09:51:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:12.198413 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af27fe5-7fd6-498d-994d-033465b3b8dd None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:12.211506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af27fe5-7fd6-498d-994d-033465b3b8dd None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:12.224786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575fc4e3-dd03-49c5-a966-7c4f48637f99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:12.225702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fc4e3-dd03-49c5-a966-7c4f48637f99 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:12.226067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fc4e3-dd03-49c5-a966-7c4f48637f99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:12.226739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fc4e3-dd03-49c5-a966-7c4f48637f99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:12.238433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575fc4e3-dd03-49c5-a966-7c4f48637f99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:12.240228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575fc4e3-dd03-49c5-a966-7c4f48637f99 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:12.253261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 683/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:13.098600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c6c3-a1f2-4129-96df-3f826bc46fa1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:13.098600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c6c3-a1f2-4129-96df-3f826bc46fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:13.098600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c6c3-a1f2-4129-96df-3f826bc46fa1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:13.099510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 694/1377] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:13.106711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8d0c2-6d20-4886-8002-19fe6b2cdd91 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:13.107402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8d0c2-6d20-4886-8002-19fe6b2cdd91 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:13.107402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8d0c2-6d20-4886-8002-19fe6b2cdd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:13.130047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-43e8d0c2-6d20-4886-8002-19fe6b2cdd91 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:13.131303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-43e8d0c2-6d20-4886-8002-19fe6b2cdd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:13.142843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 684/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:13.191148 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58c6c3-a1f2-4129-96df-3f826bc46fa1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:13.214659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58c6c3-a1f2-4129-96df-3f826bc46fa1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:13.222069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6908909-cb07-441e-a22d-c3cec960509c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:13.222765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6908909-cb07-441e-a22d-c3cec960509c service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:13.223052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6908909-cb07-441e-a22d-c3cec960509c service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:13.223328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6908909-cb07-441e-a22d-c3cec960509c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:13.233755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6908909-cb07-441e-a22d-c3cec960509c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:13.235439 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6908909-cb07-441e-a22d-c3cec960509c service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:13.239938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:51:13.248829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 695/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:15.418457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f227bc-4a28-4d2a-94e0-19086b1893e6 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:15.419104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f227bc-4a28-4d2a-94e0-19086b1893e6 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:15.419104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f227bc-4a28-4d2a-94e0-19086b1893e6 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:15.420552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 685/1380] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:15.426141 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793227-b56e-41fe-a755-df660f8177c5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:15.426639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793227-b56e-41fe-a755-df660f8177c5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:15.426753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02793227-b56e-41fe-a755-df660f8177c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:15.445916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-02793227-b56e-41fe-a755-df660f8177c5 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:15.447208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-02793227-b56e-41fe-a755-df660f8177c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:15.457083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 696/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:15.502714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f227bc-4a28-4d2a-94e0-19086b1893e6 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:15.520024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f227bc-4a28-4d2a-94e0-19086b1893e6 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:15.526310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a6a06-c487-48b9-acc9-d7cc34232d8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:15.527011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a6a06-c487-48b9-acc9-d7cc34232d8c service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:15.527352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a6a06-c487-48b9-acc9-d7cc34232d8c service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:15.527651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a6a06-c487-48b9-acc9-d7cc34232d8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:15.536711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a6a06-c487-48b9-acc9-d7cc34232d8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:15.538334 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a6a06-c487-48b9-acc9-d7cc34232d8c service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:15.549315 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 686/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:17.661172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4a7bd-abc1-4bf5-90f1-e50c20242ab8 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:17.661836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4a7bd-abc1-4bf5-90f1-e50c20242ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:17.661836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4a7bd-abc1-4bf5-90f1-e50c20242ab8 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:17.663125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 697/1383] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:17.668729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f305fd-f2f4-450c-bfdf-976d65831c18 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:17.669233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f305fd-f2f4-450c-bfdf-976d65831c18 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:17.669336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f305fd-f2f4-450c-bfdf-976d65831c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:17.688767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b6f305fd-f2f4-450c-bfdf-976d65831c18 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:17.689978 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b6f305fd-f2f4-450c-bfdf-976d65831c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:17.698438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 687/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:17.741291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4a7bd-abc1-4bf5-90f1-e50c20242ab8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:17.757564 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4a7bd-abc1-4bf5-90f1-e50c20242ab8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:17.763870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50030c6e-0696-414b-9f0d-7d1188a339f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:17.764833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030c6e-0696-414b-9f0d-7d1188a339f6 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:17.765875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030c6e-0696-414b-9f0d-7d1188a339f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:17.766236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030c6e-0696-414b-9f0d-7d1188a339f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:17.776892 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50030c6e-0696-414b-9f0d-7d1188a339f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:17.779520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50030c6e-0696-414b-9f0d-7d1188a339f6 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:17.791546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 698/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:19.914044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5819d-50ef-4daa-a801-427622ecc82f None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:19.914044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5819d-50ef-4daa-a801-427622ecc82f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:19.914919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5819d-50ef-4daa-a801-427622ecc82f None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:19.920027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 688/1386] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:19.921306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f7a5c-1c7c-4a21-8532-25ece7921c5c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:19.921919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f7a5c-1c7c-4a21-8532-25ece7921c5c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:19.921919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f7a5c-1c7c-4a21-8532-25ece7921c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:19.954298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4e3f7a5c-1c7c-4a21-8532-25ece7921c5c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:19.955525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4e3f7a5c-1c7c-4a21-8532-25ece7921c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:19.965159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 699/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:51:20.014153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5819d-50ef-4daa-a801-427622ecc82f None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:20.032985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5819d-50ef-4daa-a801-427622ecc82f None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:20.040993 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933d7a7-eb59-479c-99a5-2174d2cd750d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:20.042133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933d7a7-eb59-479c-99a5-2174d2cd750d service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:20.042398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933d7a7-eb59-479c-99a5-2174d2cd750d service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:20.042862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933d7a7-eb59-479c-99a5-2174d2cd750d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:20.052420 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933d7a7-eb59-479c-99a5-2174d2cd750d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:20.054035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933d7a7-eb59-479c-99a5-2174d2cd750d service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:20.064587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 689/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:22.185484 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bb75a-e2dd-4fe7-befe-6662c1c86d0c None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:22.186060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bb75a-e2dd-4fe7-befe-6662c1c86d0c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:22.186060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bb75a-e2dd-4fe7-befe-6662c1c86d0c None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:22.187538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 700/1389] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:22.193657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836d389-274b-490f-b0f8-815f9604cb41 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:22.194380 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836d389-274b-490f-b0f8-815f9604cb41 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:22.194380 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836d389-274b-490f-b0f8-815f9604cb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:22.213475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-0836d389-274b-490f-b0f8-815f9604cb41 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:22.214709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-0836d389-274b-490f-b0f8-815f9604cb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:22.224411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 690/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:22.270176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bb75a-e2dd-4fe7-befe-6662c1c86d0c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:22.291288 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bb75a-e2dd-4fe7-befe-6662c1c86d0c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:22.300051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504404e-74f4-468b-9a51-2aa8a3ef765c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:22.300996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504404e-74f4-468b-9a51-2aa8a3ef765c service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:22.301714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504404e-74f4-468b-9a51-2aa8a3ef765c service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:22.302086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504404e-74f4-468b-9a51-2aa8a3ef765c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:22.314633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504404e-74f4-468b-9a51-2aa8a3ef765c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:22.316326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504404e-74f4-468b-9a51-2aa8a3ef765c service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:22.327176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 701/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:24.961224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f84bc-7fea-4c60-8604-c69194c0a797 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:24.961959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f84bc-7fea-4c60-8604-c69194c0a797 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:24.961959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f84bc-7fea-4c60-8604-c69194c0a797 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:24.963107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 691/1392] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:24.970956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcaa00-2c91-4c67-80f4-23a81de42cdd None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:24.971532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcaa00-2c91-4c67-80f4-23a81de42cdd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:24.971702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcaa00-2c91-4c67-80f4-23a81de42cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:25.000977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3ddcaa00-2c91-4c67-80f4-23a81de42cdd None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:25.002877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3ddcaa00-2c91-4c67-80f4-23a81de42cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:25.015928 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 702/1393] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:25.067079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f84bc-7fea-4c60-8604-c69194c0a797 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:25.102694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f84bc-7fea-4c60-8604-c69194c0a797 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:25.112678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78bf24-d8c7-4476-86d8-8859c6670b2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:25.113405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78bf24-d8c7-4476-86d8-8859c6670b2c service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:25.113683 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78bf24-d8c7-4476-86d8-8859c6670b2c service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:25.113954 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78bf24-d8c7-4476-86d8-8859c6670b2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:25.124846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78bf24-d8c7-4476-86d8-8859c6670b2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:25.126892 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78bf24-d8c7-4476-86d8-8859c6670b2c service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:25.138230 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 692/1394] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:27.236985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ff6ec-5585-463c-ada7-e1dfc539d449 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:27.237588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ff6ec-5585-463c-ada7-e1dfc539d449 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:27.237588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ff6ec-5585-463c-ada7-e1dfc539d449 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:27.238975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 703/1395] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:27.244548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aeb8a-357a-4267-b359-3283e9a076a1 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:27.244970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aeb8a-357a-4267-b359-3283e9a076a1 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:27.245138 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aeb8a-357a-4267-b359-3283e9a076a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:27.265441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-8a7aeb8a-357a-4267-b359-3283e9a076a1 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:27.266695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8a7aeb8a-357a-4267-b359-3283e9a076a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:27.275291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 693/1396] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:27.320524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ff6ec-5585-463c-ada7-e1dfc539d449 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:27.335996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ff6ec-5585-463c-ada7-e1dfc539d449 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:27.342035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9273d-eed5-4326-a6c9-514f6b3b5458 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:27.342719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9273d-eed5-4326-a6c9-514f6b3b5458 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:27.343012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9273d-eed5-4326-a6c9-514f6b3b5458 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:27.343324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9273d-eed5-4326-a6c9-514f6b3b5458 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:27.352671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9273d-eed5-4326-a6c9-514f6b3b5458 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:27.354293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9273d-eed5-4326-a6c9-514f6b3b5458 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:27.364550 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 704/1397] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:27.678203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43308b9-0d18-49b2-bd6e-78b4b36854e4 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:27.678435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43308b9-0d18-49b2-bd6e-78b4b36854e4 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:27.678757 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43308b9-0d18-49b2-bd6e-78b4b36854e4 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:27.680238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 694/1398] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:27.686045 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cda7a-08ce-4033-b35c-e42720781461 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:27.686357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cda7a-08ce-4033-b35c-e42720781461 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:27.686661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cda7a-08ce-4033-b35c-e42720781461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:27.706074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2c2cda7a-08ce-4033-b35c-e42720781461 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:27.707321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2c2cda7a-08ce-4033-b35c-e42720781461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:27.716797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 705/1399] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:27.775443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1437dd-05b7-4106-bf5e-a186479a9a4a None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:27.775755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1437dd-05b7-4106-bf5e-a186479a9a4a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:27.776063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1437dd-05b7-4106-bf5e-a186479a9a4a None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:27.777901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 695/1400] 10.222.0.52 () {54 vars in 962 bytes} [Wed Jun 29 09:51:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:27.783644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94175013-d497-443f-9e4d-99be21def046 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:27.783904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94175013-d497-443f-9e4d-99be21def046 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:27.784218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94175013-d497-443f-9e4d-99be21def046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:27.841499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-94175013-d497-443f-9e4d-99be21def046 None None] MFA Rules not processed for user `8e850be24df24bcf972e6e31002014cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:27.842789 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-94175013-d497-443f-9e4d-99be21def046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:27.937765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 706/1401] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Jun 29 09:51:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:27.945536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1437dd-05b7-4106-bf5e-a186479a9a4a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:27.952051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=55 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:51:27.959002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1437dd-05b7-4106-bf5e-a186479a9a4a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:27.962675 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8506fd2-205f-4233-b85b-ecbbef6e05bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e850be24df24bcf972e6e31002014cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:27.963322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8506fd2-205f-4233-b85b-ecbbef6e05bc service glance] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:27.963601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8506fd2-205f-4233-b85b-ecbbef6e05bc service glance] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:27.963872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8506fd2-205f-4233-b85b-ecbbef6e05bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:27.974272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8506fd2-205f-4233-b85b-ecbbef6e05bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:27.975735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8506fd2-205f-4233-b85b-ecbbef6e05bc service glance] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:27.987973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 696/1402] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:28.727143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0a9be-9028-4319-9511-6a7b9d61afe7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:28.727687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0a9be-9028-4319-9511-6a7b9d61afe7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:28.727785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0a9be-9028-4319-9511-6a7b9d61afe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:28.778009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-13c0a9be-9028-4319-9511-6a7b9d61afe7 None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:28.779265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-13c0a9be-9028-4319-9511-6a7b9d61afe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:28.822262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:51:28.849204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 707/1403] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 09:51:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:28.890799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19455d19-d13a-4383-86d4-fa877ad17c6b None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:28.891354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19455d19-d13a-4383-86d4-fa877ad17c6b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:28.891458 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19455d19-d13a-4383-86d4-fa877ad17c6b None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:28.892858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 697/1404] 10.222.0.52 () {54 vars in 942 bytes} [Wed Jun 29 09:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:28.899568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80bf79-c794-470c-94ca-3f179e3e3d8e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:28.899857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80bf79-c794-470c-94ca-3f179e3e3d8e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:28.900132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80bf79-c794-470c-94ca-3f179e3e3d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:28.955957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8f80bf79-c794-470c-94ca-3f179e3e3d8e None None] MFA Rules not processed for user `d86d073ef5ee4631b508ab578662be96`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:28.957187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8f80bf79-c794-470c-94ca-3f179e3e3d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:29.056195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 708/1405] 10.222.0.52 () {58 vars in 1039 bytes} [Wed Jun 29 09:51:28 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:29.066660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19455d19-d13a-4383-86d4-fa877ad17c6b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:29.082592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19455d19-d13a-4383-86d4-fa877ad17c6b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:29.088085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d6f86-9bda-4648-ad14-1233b098c38f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:29.088764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d6f86-9bda-4648-ad14-1233b098c38f service swift] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:29.089044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d6f86-9bda-4648-ad14-1233b098c38f service swift] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:29.089317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d6f86-9bda-4648-ad14-1233b098c38f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:29.102498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d6f86-9bda-4648-ad14-1233b098c38f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:29.104278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d6f86-9bda-4648-ad14-1233b098c38f service swift] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:29.114017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 698/1406] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 09:51:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 09:51:29.480360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f6ff4-7ff1-42fd-962f-b71f4414f158 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:29.480644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f6ff4-7ff1-42fd-962f-b71f4414f158 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:29.483595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f6ff4-7ff1-42fd-962f-b71f4414f158 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:29.486761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 709/1407] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:51:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:29.488685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930f155-6bae-416a-a3f6-5639c93ab303 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:29.488972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930f155-6bae-416a-a3f6-5639c93ab303 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:29.489250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930f155-6bae-416a-a3f6-5639c93ab303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:29.508948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-3930f155-6bae-416a-a3f6-5639c93ab303 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:29.510534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3930f155-6bae-416a-a3f6-5639c93ab303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:29.518967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 699/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:29.564814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f6ff4-7ff1-42fd-962f-b71f4414f158 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:29.581106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f6ff4-7ff1-42fd-962f-b71f4414f158 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:29.587212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1b518-ef00-43e8-9704-4ab10394ef8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:29.587927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b518-ef00-43e8-9704-4ab10394ef8d service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:29.588260 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b518-ef00-43e8-9704-4ab10394ef8d service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:29.588476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b518-ef00-43e8-9704-4ab10394ef8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:29.598484 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b518-ef00-43e8-9704-4ab10394ef8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:29.602267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b518-ef00-43e8-9704-4ab10394ef8d service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:29.612200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 710/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:39.502670 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=14 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:51:39.607977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=59 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:51:51.531426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017401f-1dee-4c27-b54b-a7ce6dee75fc None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:51.532034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017401f-1dee-4c27-b54b-a7ce6dee75fc None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:51.532034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017401f-1dee-4c27-b54b-a7ce6dee75fc None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:51.533374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 700/1410] 10.222.0.52 () {54 vars in 918 bytes} [Wed Jun 29 09:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:51.538943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5a517-4a3e-438d-ba58-69ab5f0fe8c8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:51.539457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5a517-4a3e-438d-ba58-69ab5f0fe8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:51.539457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5a517-4a3e-438d-ba58-69ab5f0fe8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:51.559373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-95a5a517-4a3e-438d-ba58-69ab5f0fe8c8 None None] MFA Rules not processed for user `e9bff6c855ea46feaa5526d2c27c9026`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:51.560624 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-95a5a517-4a3e-438d-ba58-69ab5f0fe8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:51.571054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 711/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Jun 29 09:51:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:51.586597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3399b-054c-4ca2-bd1a-1dc07d5d692b None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:51.586917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3399b-054c-4ca2-bd1a-1dc07d5d692b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:51.587188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3399b-054c-4ca2-bd1a-1dc07d5d692b None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:51.588621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 701/1412] 10.222.0.52 () {54 vars in 918 bytes} [Wed Jun 29 09:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:51.593868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec057-8f04-47c2-8ff1-1e9bcbfdbb67 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:51.594506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec057-8f04-47c2-8ff1-1e9bcbfdbb67 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:51.594832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec057-8f04-47c2-8ff1-1e9bcbfdbb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:51.613296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-376ec057-8f04-47c2-8ff1-1e9bcbfdbb67 None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:51.614510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-376ec057-8f04-47c2-8ff1-1e9bcbfdbb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:51.624761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 712/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Jun 29 09:51:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:54.212476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc542883-a1ed-4dd3-9bc2-1288bc69bd56 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:54.213085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc542883-a1ed-4dd3-9bc2-1288bc69bd56 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:54.213085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc542883-a1ed-4dd3-9bc2-1288bc69bd56 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:54.214402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:54.220976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055f074-e094-429c-98cf-e4f0f5788367 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:54.221431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055f074-e094-429c-98cf-e4f0f5788367 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:54.221580 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055f074-e094-429c-98cf-e4f0f5788367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:54.241699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9055f074-e094-429c-98cf-e4f0f5788367 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:54.242919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9055f074-e094-429c-98cf-e4f0f5788367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:54.251393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:51:54.295419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc542883-a1ed-4dd3-9bc2-1288bc69bd56 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:54.312252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc542883-a1ed-4dd3-9bc2-1288bc69bd56 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:54.318001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc74a16-576b-403d-bc5f-7799d463f21b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:54.318757 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc74a16-576b-403d-bc5f-7799d463f21b service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:54.319039 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc74a16-576b-403d-bc5f-7799d463f21b service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:54.319314 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc74a16-576b-403d-bc5f-7799d463f21b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:54.328400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc74a16-576b-403d-bc5f-7799d463f21b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:54.330034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc74a16-576b-403d-bc5f-7799d463f21b service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:54.340324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 703/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:55.248568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc329b-e6f9-47e9-b294-fea3b612028d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:55.249173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc329b-e6f9-47e9-b294-fea3b612028d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:55.249173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc329b-e6f9-47e9-b294-fea3b612028d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:55.250505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:55.255828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8eeded-10da-4a5f-b2f5-7dafa99baef2 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:55.256296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8eeded-10da-4a5f-b2f5-7dafa99baef2 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:55.256399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8eeded-10da-4a5f-b2f5-7dafa99baef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:55.276140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-cd8eeded-10da-4a5f-b2f5-7dafa99baef2 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:55.277960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cd8eeded-10da-4a5f-b2f5-7dafa99baef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:55.286675 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:55.448114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc329b-e6f9-47e9-b294-fea3b612028d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:55.464248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc329b-e6f9-47e9-b294-fea3b612028d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:55.470784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb558e-4513-403e-a2ce-adba71846c84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:55.471423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb558e-4513-403e-a2ce-adba71846c84 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:55.471703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb558e-4513-403e-a2ce-adba71846c84 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:55.471961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb558e-4513-403e-a2ce-adba71846c84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:55.480824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb558e-4513-403e-a2ce-adba71846c84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:55.482466 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb558e-4513-403e-a2ce-adba71846c84 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:55.492893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 715/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:56.375211 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f28f3a-742f-48bd-9fca-499751316693 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:56.375842 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f28f3a-742f-48bd-9fca-499751316693 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:56.375842 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f28f3a-742f-48bd-9fca-499751316693 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:56.377155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:56.382322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697ff2d-ba31-4d69-930b-2ae37fdd5d8a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:56.382764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697ff2d-ba31-4d69-930b-2ae37fdd5d8a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:56.382911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697ff2d-ba31-4d69-930b-2ae37fdd5d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:56.402671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6697ff2d-ba31-4d69-930b-2ae37fdd5d8a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:56.403877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6697ff2d-ba31-4d69-930b-2ae37fdd5d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:56.413220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:56.570083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f28f3a-742f-48bd-9fca-499751316693 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:56.586278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f28f3a-742f-48bd-9fca-499751316693 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:56.593096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c17faf-4197-4249-8c7a-fd1409c020ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:56.593748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c17faf-4197-4249-8c7a-fd1409c020ba service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:56.594026 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c17faf-4197-4249-8c7a-fd1409c020ba service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:56.594689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c17faf-4197-4249-8c7a-fd1409c020ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:56.603356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c17faf-4197-4249-8c7a-fd1409c020ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:56.605096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c17faf-4197-4249-8c7a-fd1409c020ba service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:56.615197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 706/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:57.507640 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7c7e7-2738-4be5-afa6-adad44a74330 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:57.508229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7c7e7-2738-4be5-afa6-adad44a74330 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:57.508229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7c7e7-2738-4be5-afa6-adad44a74330 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:57.510238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:51:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:57.517024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850de4-643b-4b27-9536-1d8164e97c22 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:57.517579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850de4-643b-4b27-9536-1d8164e97c22 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:57.517579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850de4-643b-4b27-9536-1d8164e97c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:57.539610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-53850de4-643b-4b27-9536-1d8164e97c22 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:57.540822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-53850de4-643b-4b27-9536-1d8164e97c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:57.550099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:51:57.701301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7c7e7-2738-4be5-afa6-adad44a74330 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:57.718365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7c7e7-2738-4be5-afa6-adad44a74330 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:57.725282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2364a0-65c4-42c9-96de-3cf4814d23e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:57.725942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2364a0-65c4-42c9-96de-3cf4814d23e2 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:57.726221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2364a0-65c4-42c9-96de-3cf4814d23e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:57.726500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2364a0-65c4-42c9-96de-3cf4814d23e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:57.735927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2364a0-65c4-42c9-96de-3cf4814d23e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:57.737614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2364a0-65c4-42c9-96de-3cf4814d23e2 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:57.748954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 718/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:58.629884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e6782-72fd-4054-806d-e59d75c9bc78 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:58.630622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e6782-72fd-4054-806d-e59d75c9bc78 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:58.630622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e6782-72fd-4054-806d-e59d75c9bc78 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:58.631876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:58.637733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33f42a-9d0c-4053-8a71-0f588fdb3f87 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:58.638214 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33f42a-9d0c-4053-8a71-0f588fdb3f87 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:58.638319 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33f42a-9d0c-4053-8a71-0f588fdb3f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:58.657285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2a33f42a-9d0c-4053-8a71-0f588fdb3f87 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:58.658464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2a33f42a-9d0c-4053-8a71-0f588fdb3f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:58.667224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:51:58.833892 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e6782-72fd-4054-806d-e59d75c9bc78 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:58.852936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e6782-72fd-4054-806d-e59d75c9bc78 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:58.860066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcdfcf-a4d2-4256-a818-ed8a781dabae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:58.860710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcdfcf-a4d2-4256-a818-ed8a781dabae service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:58.860998 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcdfcf-a4d2-4256-a818-ed8a781dabae service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:58.861278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcdfcf-a4d2-4256-a818-ed8a781dabae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:58.870326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcdfcf-a4d2-4256-a818-ed8a781dabae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:58.871988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcdfcf-a4d2-4256-a818-ed8a781dabae service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:58.882112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 709/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:51:59.692546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826a757-a6d3-40ef-b866-e6ce3b8e2223 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:59.693502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826a757-a6d3-40ef-b866-e6ce3b8e2223 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:59.693502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826a757-a6d3-40ef-b866-e6ce3b8e2223 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:59.695658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:51:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:51:59.700855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576ed1d-3dad-4ef5-857b-045ae2863a30 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:59.701333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576ed1d-3dad-4ef5-857b-045ae2863a30 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:59.701457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576ed1d-3dad-4ef5-857b-045ae2863a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:59.720337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-4576ed1d-3dad-4ef5-857b-045ae2863a30 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:51:59.721548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4576ed1d-3dad-4ef5-857b-045ae2863a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:51:59.730141 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:51:59.883043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826a757-a6d3-40ef-b866-e6ce3b8e2223 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:51:59.899790 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826a757-a6d3-40ef-b866-e6ce3b8e2223 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:51:59.906035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295efeb3-1f6b-4f9f-83fa-a05d19b48622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:51:59.906732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295efeb3-1f6b-4f9f-83fa-a05d19b48622 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:51:59.907011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295efeb3-1f6b-4f9f-83fa-a05d19b48622 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:51:59.907284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295efeb3-1f6b-4f9f-83fa-a05d19b48622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:51:59.917403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295efeb3-1f6b-4f9f-83fa-a05d19b48622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:51:59.918510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295efeb3-1f6b-4f9f-83fa-a05d19b48622 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:51:59.928654 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 721/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:00.764909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9259e99-e0fb-4412-b670-f010452a0707 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:00.765541 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9259e99-e0fb-4412-b670-f010452a0707 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:00.765541 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9259e99-e0fb-4412-b670-f010452a0707 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:00.767115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:00.772796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cce09a-eb84-41e0-b936-73a99ffd9cc0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:00.773365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cce09a-eb84-41e0-b936-73a99ffd9cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:00.773365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cce09a-eb84-41e0-b936-73a99ffd9cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:00.793261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f2cce09a-eb84-41e0-b936-73a99ffd9cc0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:00.794500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f2cce09a-eb84-41e0-b936-73a99ffd9cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:00.803101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:00.953373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9259e99-e0fb-4412-b670-f010452a0707 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:00.970386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9259e99-e0fb-4412-b670-f010452a0707 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:00.976682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ccb07-56e5-427f-94fc-1a89bf24da67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:00.977345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ccb07-56e5-427f-94fc-1a89bf24da67 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:00.977619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ccb07-56e5-427f-94fc-1a89bf24da67 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:00.977922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ccb07-56e5-427f-94fc-1a89bf24da67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:00.988087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ccb07-56e5-427f-94fc-1a89bf24da67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:00.989694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ccb07-56e5-427f-94fc-1a89bf24da67 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:01.003914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 712/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:01.839772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ced9e5-8d59-486b-9090-d0b42857ea30 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:01.840550 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ced9e5-8d59-486b-9090-d0b42857ea30 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:01.840935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ced9e5-8d59-486b-9090-d0b42857ea30 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:01.842456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:52:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:01.851355 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a1a01-2dfc-494e-9277-e56e45aec2fd None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:01.851903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a1a01-2dfc-494e-9277-e56e45aec2fd None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:01.852326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a1a01-2dfc-494e-9277-e56e45aec2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:01.881386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-7d4a1a01-2dfc-494e-9277-e56e45aec2fd None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:01.883012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7d4a1a01-2dfc-494e-9277-e56e45aec2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:01.894700 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:02.167555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ced9e5-8d59-486b-9090-d0b42857ea30 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:02.183497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ced9e5-8d59-486b-9090-d0b42857ea30 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:02.190511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c27de-2497-486c-985c-2a58a3aa2a8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:02.191224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c27de-2497-486c-985c-2a58a3aa2a8b service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:02.191506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c27de-2497-486c-985c-2a58a3aa2a8b service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:02.191784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c27de-2497-486c-985c-2a58a3aa2a8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:02.201638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c27de-2497-486c-985c-2a58a3aa2a8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:02.203301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c27de-2497-486c-985c-2a58a3aa2a8b service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:02.214299 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 724/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:03.054908 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e605e06-0e3f-42cb-8d63-c3785317d668 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:03.055628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e605e06-0e3f-42cb-8d63-c3785317d668 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:03.056340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e605e06-0e3f-42cb-8d63-c3785317d668 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:03.058720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:52:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:03.064322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016ffe3-b3d4-4e35-9e33-0bf6e3ae5b18 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:03.064828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016ffe3-b3d4-4e35-9e33-0bf6e3ae5b18 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:03.064961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016ffe3-b3d4-4e35-9e33-0bf6e3ae5b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:03.084554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c016ffe3-b3d4-4e35-9e33-0bf6e3ae5b18 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:03.085785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c016ffe3-b3d4-4e35-9e33-0bf6e3ae5b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:03.095247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:03.247970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e605e06-0e3f-42cb-8d63-c3785317d668 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:03.271230 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e605e06-0e3f-42cb-8d63-c3785317d668 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:03.280012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36270f-09a6-4697-bb66-40bb754851bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:03.281107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36270f-09a6-4697-bb66-40bb754851bc service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:03.281581 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36270f-09a6-4697-bb66-40bb754851bc service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:03.282045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36270f-09a6-4697-bb66-40bb754851bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:03.297844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36270f-09a6-4697-bb66-40bb754851bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:03.300894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36270f-09a6-4697-bb66-40bb754851bc service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:03.320317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 715/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:05.518052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb6975-6bdd-482f-964c-0f29c4d7c958 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:05.518695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb6975-6bdd-482f-964c-0f29c4d7c958 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:05.518695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb6975-6bdd-482f-964c-0f29c4d7c958 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:05.520147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:05.527180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849d008-18dd-43ce-8bde-d232472f05a6 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:05.527997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849d008-18dd-43ce-8bde-d232472f05a6 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:05.528332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849d008-18dd-43ce-8bde-d232472f05a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:05.548339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-8849d008-18dd-43ce-8bde-d232472f05a6 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:05.549663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8849d008-18dd-43ce-8bde-d232472f05a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:05.558252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:05.600065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb6975-6bdd-482f-964c-0f29c4d7c958 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:05.614689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb6975-6bdd-482f-964c-0f29c4d7c958 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:05.620806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31819803-790a-4471-b41d-acd0f35e2481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:05.622976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31819803-790a-4471-b41d-acd0f35e2481 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:05.622976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31819803-790a-4471-b41d-acd0f35e2481 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:05.622976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31819803-790a-4471-b41d-acd0f35e2481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:05.633729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31819803-790a-4471-b41d-acd0f35e2481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:05.635264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31819803-790a-4471-b41d-acd0f35e2481 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:05.646126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 727/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:07.988203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7f9f7-b5f1-4e5f-b1db-08a8f50ee56a None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:07.989113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7f9f7-b5f1-4e5f-b1db-08a8f50ee56a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:07.989113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7f9f7-b5f1-4e5f-b1db-08a8f50ee56a None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:07.990930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:52:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:07.998758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44934b91-8b62-4a85-944b-104debbfcdb9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:07.999379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44934b91-8b62-4a85-944b-104debbfcdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:07.999379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44934b91-8b62-4a85-944b-104debbfcdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:08.020470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-44934b91-8b62-4a85-944b-104debbfcdb9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:08.021883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-44934b91-8b62-4a85-944b-104debbfcdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:08.032346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:08.074888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7f9f7-b5f1-4e5f-b1db-08a8f50ee56a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:08.087409 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7f9f7-b5f1-4e5f-b1db-08a8f50ee56a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:08.091975 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38799ab-4b3d-4d1f-b2d2-2a46b55c8e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:08.092653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38799ab-4b3d-4d1f-b2d2-2a46b55c8e46 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:08.092958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38799ab-4b3d-4d1f-b2d2-2a46b55c8e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:08.093232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38799ab-4b3d-4d1f-b2d2-2a46b55c8e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:08.104093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38799ab-4b3d-4d1f-b2d2-2a46b55c8e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:08.105600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38799ab-4b3d-4d1f-b2d2-2a46b55c8e46 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:08.117042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 718/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:10.782727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c45a-4ff7-4e95-bd67-df7aec1ed1e6 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:10.783355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c45a-4ff7-4e95-bd67-df7aec1ed1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:10.783739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c45a-4ff7-4e95-bd67-df7aec1ed1e6 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:10.786381 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:52:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:10.792895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2c91-b816-4f25-b50c-ba44c82062bc None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:10.793540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2c91-b816-4f25-b50c-ba44c82062bc None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:10.793540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2c91-b816-4f25-b50c-ba44c82062bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:10.817195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-423e2c91-b816-4f25-b50c-ba44c82062bc None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:10.818452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-423e2c91-b816-4f25-b50c-ba44c82062bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:10.829003 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:10.874281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436c45a-4ff7-4e95-bd67-df7aec1ed1e6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:10.896337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436c45a-4ff7-4e95-bd67-df7aec1ed1e6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:10.902820 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a588aee-bc20-473a-aec5-e92de270cd71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:10.903501 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a588aee-bc20-473a-aec5-e92de270cd71 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:10.903813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a588aee-bc20-473a-aec5-e92de270cd71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:10.904089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a588aee-bc20-473a-aec5-e92de270cd71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:10.915691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a588aee-bc20-473a-aec5-e92de270cd71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:10.918315 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a588aee-bc20-473a-aec5-e92de270cd71 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:10.935500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 730/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:14.070364 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef4bc8-fe3e-4e18-83c0-6aaa82ec10b1 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:14.070980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef4bc8-fe3e-4e18-83c0-6aaa82ec10b1 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:14.071181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef4bc8-fe3e-4e18-83c0-6aaa82ec10b1 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:14.072779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:52:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:14.081620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69e5af-dfc8-4c1b-acc3-f857c093a497 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:14.082251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69e5af-dfc8-4c1b-acc3-f857c093a497 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:14.082356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69e5af-dfc8-4c1b-acc3-f857c093a497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:14.103617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-bc69e5af-dfc8-4c1b-acc3-f857c093a497 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:14.104928 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bc69e5af-dfc8-4c1b-acc3-f857c093a497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:14.114437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:14.162046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef4bc8-fe3e-4e18-83c0-6aaa82ec10b1 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:14.176248 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef4bc8-fe3e-4e18-83c0-6aaa82ec10b1 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:14.180523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa46d74-0552-43fc-a6cb-7de77523a0a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:14.181244 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa46d74-0552-43fc-a6cb-7de77523a0a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:14.181578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa46d74-0552-43fc-a6cb-7de77523a0a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:14.181836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa46d74-0552-43fc-a6cb-7de77523a0a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:14.193358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa46d74-0552-43fc-a6cb-7de77523a0a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:14.195067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa46d74-0552-43fc-a6cb-7de77523a0a3 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:14.206420 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 721/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:16.490773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c03fd-6e5a-4484-a516-725bb2a4ed4d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:16.491363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c03fd-6e5a-4484-a516-725bb2a4ed4d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:16.491809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c03fd-6e5a-4484-a516-725bb2a4ed4d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:16.494506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:52:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:16.500362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685b50a-75da-4c1a-9362-13f12d85967d None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:16.500865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685b50a-75da-4c1a-9362-13f12d85967d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:16.500955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685b50a-75da-4c1a-9362-13f12d85967d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:16.522631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2685b50a-75da-4c1a-9362-13f12d85967d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:16.524148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2685b50a-75da-4c1a-9362-13f12d85967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:16.534320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:16.578406 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c03fd-6e5a-4484-a516-725bb2a4ed4d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:16.590042 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c03fd-6e5a-4484-a516-725bb2a4ed4d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:16.593816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed7fd6-e97b-478d-ad8d-7e18a61e0f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:16.595094 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed7fd6-e97b-478d-ad8d-7e18a61e0f42 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:16.595373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed7fd6-e97b-478d-ad8d-7e18a61e0f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:16.595644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed7fd6-e97b-478d-ad8d-7e18a61e0f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:16.606793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed7fd6-e97b-478d-ad8d-7e18a61e0f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:16.608260 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed7fd6-e97b-478d-ad8d-7e18a61e0f42 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:16.619177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 733/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:18.952735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0db2c-3658-46ef-b9d4-91defd7262e0 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:18.953513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0db2c-3658-46ef-b9d4-91defd7262e0 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:18.953513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0db2c-3658-46ef-b9d4-91defd7262e0 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:18.955697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:52:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:18.961974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319c49f-4054-41d6-a6c8-ca38408793e0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:18.962513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319c49f-4054-41d6-a6c8-ca38408793e0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:18.962668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319c49f-4054-41d6-a6c8-ca38408793e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:18.984546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d319c49f-4054-41d6-a6c8-ca38408793e0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:18.985918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d319c49f-4054-41d6-a6c8-ca38408793e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:18.995484 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:19.035574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0db2c-3658-46ef-b9d4-91defd7262e0 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:19.049004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0db2c-3658-46ef-b9d4-91defd7262e0 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:19.052778 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81ed64-94e0-475b-8b4f-dc3e98a832be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:19.053465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81ed64-94e0-475b-8b4f-dc3e98a832be service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:19.053724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81ed64-94e0-475b-8b4f-dc3e98a832be service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:19.054014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81ed64-94e0-475b-8b4f-dc3e98a832be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:19.064978 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81ed64-94e0-475b-8b4f-dc3e98a832be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:19.066637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81ed64-94e0-475b-8b4f-dc3e98a832be service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:19.078038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 724/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:21.276359 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b79f-57ec-45c1-854f-cded6d343193 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:21.276957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b79f-57ec-45c1-854f-cded6d343193 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:21.276957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b79f-57ec-45c1-854f-cded6d343193 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:21.278595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:21.284596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552b621-55b0-461f-b431-a2aa2f5b9583 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:21.285396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552b621-55b0-461f-b431-a2aa2f5b9583 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:21.285396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552b621-55b0-461f-b431-a2aa2f5b9583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:21.291848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=70 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:52:21.319176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2552b621-55b0-461f-b431-a2aa2f5b9583 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:21.320408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2552b621-55b0-461f-b431-a2aa2f5b9583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:21.330547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:52:21.369595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8b79f-57ec-45c1-854f-cded6d343193 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:21.375756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=73 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:52:21.382734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8b79f-57ec-45c1-854f-cded6d343193 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:21.386721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fc8e3-f7cc-423d-ae41-aaf0a74a8660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:21.387368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fc8e3-f7cc-423d-ae41-aaf0a74a8660 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:21.387629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fc8e3-f7cc-423d-ae41-aaf0a74a8660 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:21.387898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fc8e3-f7cc-423d-ae41-aaf0a74a8660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:21.398194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fc8e3-f7cc-423d-ae41-aaf0a74a8660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:21.399698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fc8e3-f7cc-423d-ae41-aaf0a74a8660 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:21.410389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 736/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:23.702229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cb0ef-5e9b-41ac-87bc-5ed6c472bc0d None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:23.702942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cb0ef-5e9b-41ac-87bc-5ed6c472bc0d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:23.703220 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cb0ef-5e9b-41ac-87bc-5ed6c472bc0d None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:23.705834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:52:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:23.711854 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead545b7-d514-4e62-8d92-edf9a773e426 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:23.712326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead545b7-d514-4e62-8d92-edf9a773e426 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:23.712657 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead545b7-d514-4e62-8d92-edf9a773e426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:23.734474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ead545b7-d514-4e62-8d92-edf9a773e426 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:23.735733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ead545b7-d514-4e62-8d92-edf9a773e426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:23.746427 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:23.793965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cb0ef-5e9b-41ac-87bc-5ed6c472bc0d None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:23.806747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cb0ef-5e9b-41ac-87bc-5ed6c472bc0d None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:23.811614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6930c7f-ddba-48c8-bae0-e1bbf2ef9639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:23.812273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6930c7f-ddba-48c8-bae0-e1bbf2ef9639 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:23.812546 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6930c7f-ddba-48c8-bae0-e1bbf2ef9639 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:23.812812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6930c7f-ddba-48c8-bae0-e1bbf2ef9639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:23.823441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6930c7f-ddba-48c8-bae0-e1bbf2ef9639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:23.825530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6930c7f-ddba-48c8-bae0-e1bbf2ef9639 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:23.836633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 727/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:26.261833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77b7ff-f2b9-4b46-a31c-f9d6282065e1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:26.263197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77b7ff-f2b9-4b46-a31c-f9d6282065e1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:26.263197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77b7ff-f2b9-4b46-a31c-f9d6282065e1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:26.264003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:26.270377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd030c4-feb9-4da6-9bd4-b252c3a91dd6 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:26.271003 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd030c4-feb9-4da6-9bd4-b252c3a91dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:26.271204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd030c4-feb9-4da6-9bd4-b252c3a91dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:26.291430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-dcd030c4-feb9-4da6-9bd4-b252c3a91dd6 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:26.293404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-dcd030c4-feb9-4da6-9bd4-b252c3a91dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:26.302116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:52:26.340378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77b7ff-f2b9-4b46-a31c-f9d6282065e1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:26.352950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77b7ff-f2b9-4b46-a31c-f9d6282065e1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:26.357646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ee6e1-50f6-404f-9b43-4ba86d9c5863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:26.358306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ee6e1-50f6-404f-9b43-4ba86d9c5863 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:26.358608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ee6e1-50f6-404f-9b43-4ba86d9c5863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:26.358886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ee6e1-50f6-404f-9b43-4ba86d9c5863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:26.369168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ee6e1-50f6-404f-9b43-4ba86d9c5863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:26.371219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ee6e1-50f6-404f-9b43-4ba86d9c5863 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:26.381713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 739/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:28.483461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99b716-260f-48f1-8811-b7dc526f72dd None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:28.484103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99b716-260f-48f1-8811-b7dc526f72dd None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:28.484103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99b716-260f-48f1-8811-b7dc526f72dd None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:28.485400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:28.490944 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4238fce-0e84-4a15-a816-7e28506c7102 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:28.491432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4238fce-0e84-4a15-a816-7e28506c7102 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:28.491527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4238fce-0e84-4a15-a816-7e28506c7102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:28.509886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d4238fce-0e84-4a15-a816-7e28506c7102 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:28.511132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d4238fce-0e84-4a15-a816-7e28506c7102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:28.519592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:28.556485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99b716-260f-48f1-8811-b7dc526f72dd None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:28.568944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99b716-260f-48f1-8811-b7dc526f72dd None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:28.572959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22474936-3bef-4e88-9052-15e45b89c051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:28.573650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22474936-3bef-4e88-9052-15e45b89c051 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:28.573933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22474936-3bef-4e88-9052-15e45b89c051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:28.574201 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22474936-3bef-4e88-9052-15e45b89c051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:28.583685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22474936-3bef-4e88-9052-15e45b89c051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:28.585143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22474936-3bef-4e88-9052-15e45b89c051 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:28.595185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 730/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:38.505132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=17 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:52:38.590581 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=17 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:52:48.627289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf79432-b015-487b-8567-3e5f346ce167 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:48.627921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf79432-b015-487b-8567-3e5f346ce167 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:48.627921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf79432-b015-487b-8567-3e5f346ce167 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:48.629220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 741/1471] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:48.635357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87954-345e-4078-aec4-6d4fd3d3d2a3 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:48.636333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87954-345e-4078-aec4-6d4fd3d3d2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:48.636838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e87954-345e-4078-aec4-6d4fd3d3d2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:48.656460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-f1e87954-345e-4078-aec4-6d4fd3d3d2a3 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:48.657687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f1e87954-345e-4078-aec4-6d4fd3d3d2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:48.666340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 731/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:48.715608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1ea79-5194-49ea-ad7a-22b39bf2279a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:48.715874 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1ea79-5194-49ea-ad7a-22b39bf2279a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:48.716181 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1ea79-5194-49ea-ad7a-22b39bf2279a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:48.717591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 742/1473] 10.222.0.52 () {54 vars in 962 bytes} [Wed Jun 29 09:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:48.723310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb30fd-b81d-4ed0-9bca-fb10b1a0f10c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:48.723619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb30fd-b81d-4ed0-9bca-fb10b1a0f10c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:48.723899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb30fd-b81d-4ed0-9bca-fb10b1a0f10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:48.749038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-12fb30fd-b81d-4ed0-9bca-fb10b1a0f10c None None] MFA Rules not processed for user `8e850be24df24bcf972e6e31002014cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:48.750247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-12fb30fd-b81d-4ed0-9bca-fb10b1a0f10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:48.757276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 732/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Jun 29 09:52:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:48.764980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1ea79-5194-49ea-ad7a-22b39bf2279a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:48.778830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1ea79-5194-49ea-ad7a-22b39bf2279a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:48.782697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f1627-d019-498b-8b7e-c7317e1c0937 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e850be24df24bcf972e6e31002014cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:48.783345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f1627-d019-498b-8b7e-c7317e1c0937 service glance] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:48.783616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f1627-d019-498b-8b7e-c7317e1c0937 service glance] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:48.783884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f1627-d019-498b-8b7e-c7317e1c0937 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:48.799313 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f1627-d019-498b-8b7e-c7317e1c0937 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:48.801147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f1627-d019-498b-8b7e-c7317e1c0937 service glance] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:48.813691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 743/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:50.963125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9249104-066f-4fd6-aefb-a844d64fb842 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:50.964056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9249104-066f-4fd6-aefb-a844d64fb842 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:50.964056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9249104-066f-4fd6-aefb-a844d64fb842 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:50.965254 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 733/1476] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:50.971533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292187a-2572-4a36-9d4d-66b1fd9de4fa None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:50.972225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292187a-2572-4a36-9d4d-66b1fd9de4fa None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:50.972432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292187a-2572-4a36-9d4d-66b1fd9de4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:50.992190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8292187a-2572-4a36-9d4d-66b1fd9de4fa None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:50.993401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8292187a-2572-4a36-9d4d-66b1fd9de4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:51.003297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 744/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:51.027685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9249104-066f-4fd6-aefb-a844d64fb842 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:51.043083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9249104-066f-4fd6-aefb-a844d64fb842 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:51.048980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842ed89-4906-4164-8c44-b26267a6c177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:51.049647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842ed89-4906-4164-8c44-b26267a6c177 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:51.049921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842ed89-4906-4164-8c44-b26267a6c177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:51.050188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842ed89-4906-4164-8c44-b26267a6c177 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:51.050979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842ed89-4906-4164-8c44-b26267a6c177 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:51.052793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: WARNING py.warnings [None req-a842ed89-4906-4164-8c44-b26267a6c177 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:52:51.052793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: warnings.warn(msg) Jun 29 09:52:51.052793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: Jun 29 09:52:51.053954 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842ed89-4906-4164-8c44-b26267a6c177 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:51.074986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 734/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 09:52:51 2022] GET /identity/v3/projects => generated 2553 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:52:53.266148 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7fef5-9650-4891-995c-55e5762a3691 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:53.266783 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7fef5-9650-4891-995c-55e5762a3691 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:53.267201 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7fef5-9650-4891-995c-55e5762a3691 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:53.270693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 745/1479] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:52:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:53.276403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586abf0-c3ff-4a47-a566-e2cd50b09f19 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:53.277241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586abf0-c3ff-4a47-a566-e2cd50b09f19 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:53.277241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586abf0-c3ff-4a47-a566-e2cd50b09f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:53.302759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b586abf0-c3ff-4a47-a566-e2cd50b09f19 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:53.304489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b586abf0-c3ff-4a47-a566-e2cd50b09f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:53.318495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 735/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:52:53.365594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7fef5-9650-4891-995c-55e5762a3691 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:53.382982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7fef5-9650-4891-995c-55e5762a3691 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:53.391892 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4cbe0-087d-4ce9-b0c3-8e03ef31cc79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:53.392923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4cbe0-087d-4ce9-b0c3-8e03ef31cc79 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:53.393317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4cbe0-087d-4ce9-b0c3-8e03ef31cc79 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:53.393725 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4cbe0-087d-4ce9-b0c3-8e03ef31cc79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:53.409203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b4cbe0-087d-4ce9-b0c3-8e03ef31cc79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:53.411677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b4cbe0-087d-4ce9-b0c3-8e03ef31cc79 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:53.431306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 746/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:55.858408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef4c99-3c58-4f70-b231-d0d4bbc446f0 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:55.859089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef4c99-3c58-4f70-b231-d0d4bbc446f0 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:55.859089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef4c99-3c58-4f70-b231-d0d4bbc446f0 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:55.860510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 736/1482] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:55.866175 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a241b-82c7-4cb9-b37e-bd744aaed536 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:55.866721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a241b-82c7-4cb9-b37e-bd744aaed536 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:55.866721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a241b-82c7-4cb9-b37e-bd744aaed536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:55.887255 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5e0a241b-82c7-4cb9-b37e-bd744aaed536 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:55.888563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5e0a241b-82c7-4cb9-b37e-bd744aaed536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:55.897586 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 747/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:52:55.939864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef4c99-3c58-4f70-b231-d0d4bbc446f0 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:55.956027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef4c99-3c58-4f70-b231-d0d4bbc446f0 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:55.960458 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815812b6-265a-4f11-9540-ffb0d6332fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:55.961194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815812b6-265a-4f11-9540-ffb0d6332fec service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:55.961477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815812b6-265a-4f11-9540-ffb0d6332fec service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:55.961788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815812b6-265a-4f11-9540-ffb0d6332fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:55.973430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815812b6-265a-4f11-9540-ffb0d6332fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:55.974987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815812b6-265a-4f11-9540-ffb0d6332fec service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:55.988618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 737/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:52:58.188771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd511f-e96d-49a6-9ea9-89b3e77084e5 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:58.189352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd511f-e96d-49a6-9ea9-89b3e77084e5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:58.189352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd511f-e96d-49a6-9ea9-89b3e77084e5 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:58.190740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 748/1485] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:52:58.197162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf4777-4559-4ef0-b65d-f1b1a98edda2 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:58.197651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf4777-4559-4ef0-b65d-f1b1a98edda2 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:58.197873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf4777-4559-4ef0-b65d-f1b1a98edda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:58.220700 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-3ddf4777-4559-4ef0-b65d-f1b1a98edda2 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:52:58.221972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3ddf4777-4559-4ef0-b65d-f1b1a98edda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:52:58.230745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 738/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:52:58.271236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd511f-e96d-49a6-9ea9-89b3e77084e5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:52:58.286549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd511f-e96d-49a6-9ea9-89b3e77084e5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:52:58.291047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98188db7-e123-46b4-8a6c-6898d87845f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:52:58.291873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98188db7-e123-46b4-8a6c-6898d87845f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:52:58.292291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98188db7-e123-46b4-8a6c-6898d87845f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:52:58.292847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98188db7-e123-46b4-8a6c-6898d87845f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:52:58.306534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98188db7-e123-46b4-8a6c-6898d87845f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:52:58.309071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98188db7-e123-46b4-8a6c-6898d87845f1 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:52:58.324752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 749/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:53:00.728169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69fceb-a354-44de-b51e-ab285322bb39 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:53:00.728867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69fceb-a354-44de-b51e-ab285322bb39 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:53:00.728867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69fceb-a354-44de-b51e-ab285322bb39 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:53:00.732692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 739/1488] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:53:00.736498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa6a54-6b43-421e-b08a-7e01e9311045 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:53:00.737029 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa6a54-6b43-421e-b08a-7e01e9311045 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:53:00.737136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa6a54-6b43-421e-b08a-7e01e9311045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:53:00.757429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-78fa6a54-6b43-421e-b08a-7e01e9311045 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:53:00.758974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-78fa6a54-6b43-421e-b08a-7e01e9311045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:53:00.769552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 750/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:53:00.817910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69fceb-a354-44de-b51e-ab285322bb39 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:53:00.830558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69fceb-a354-44de-b51e-ab285322bb39 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:53:00.835041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075514d-ef76-45e8-aeb1-fde86558680d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:53:00.835701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075514d-ef76-45e8-aeb1-fde86558680d service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:53:00.835978 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075514d-ef76-45e8-aeb1-fde86558680d service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:53:00.836256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075514d-ef76-45e8-aeb1-fde86558680d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:53:00.847257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075514d-ef76-45e8-aeb1-fde86558680d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:53:00.848720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075514d-ef76-45e8-aeb1-fde86558680d service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:53:00.859296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 740/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:53:03.351185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3bcf-b800-4c94-8362-b1e1a59701be None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:53:03.351747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3bcf-b800-4c94-8362-b1e1a59701be None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:53:03.351747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3bcf-b800-4c94-8362-b1e1a59701be None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:53:03.353040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 751/1491] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 09:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:53:03.358599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabed40e-c78e-4fe4-b5bc-ef7b8936b349 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:53:03.359052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabed40e-c78e-4fe4-b5bc-ef7b8936b349 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:53:03.359156 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabed40e-c78e-4fe4-b5bc-ef7b8936b349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:53:03.378528 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fabed40e-c78e-4fe4-b5bc-ef7b8936b349 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:53:03.379816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fabed40e-c78e-4fe4-b5bc-ef7b8936b349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:53:03.390407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 741/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 09:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:53:03.431939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae3bcf-b800-4c94-8362-b1e1a59701be None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:53:03.448602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae3bcf-b800-4c94-8362-b1e1a59701be None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:53:03.453089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fef52a-397d-4929-972a-064ddf759433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:53:03.453735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fef52a-397d-4929-972a-064ddf759433 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:53:03.454008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fef52a-397d-4929-972a-064ddf759433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:53:03.454274 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fef52a-397d-4929-972a-064ddf759433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:53:03.464561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fef52a-397d-4929-972a-064ddf759433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:53:03.466027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fef52a-397d-4929-972a-064ddf759433 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:53:03.476374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 752/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:53:13.373111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=33 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:53:13.471894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=33 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:54:37.775297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc15fc4-fdcb-4afd-9810-5450cd3754bf None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:37.775945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc15fc4-fdcb-4afd-9810-5450cd3754bf None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:37.775945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc15fc4-fdcb-4afd-9810-5450cd3754bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:37.795943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fdc15fc4-fdcb-4afd-9810-5450cd3754bf None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:54:37.797149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fdc15fc4-fdcb-4afd-9810-5450cd3754bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:54:37.804852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 742/1494] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:54:37.813556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf586b-ded1-4853-a59a-a0c1507ca0fb None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:37.814093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf586b-ded1-4853-a59a-a0c1507ca0fb None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:37.814093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf586b-ded1-4853-a59a-a0c1507ca0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:37.833314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-40cf586b-ded1-4853-a59a-a0c1507ca0fb None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:54:37.834528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-40cf586b-ded1-4853-a59a-a0c1507ca0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:54:37.845805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 753/1495] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:54:37.850990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc15fc4-fdcb-4afd-9810-5450cd3754bf None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:54:37.863480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc15fc4-fdcb-4afd-9810-5450cd3754bf None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:54:37.869363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a5c6f-d3eb-4e27-abeb-ec19c9e214cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:54:37.870041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a5c6f-d3eb-4e27-abeb-ec19c9e214cb admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:37.870423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a5c6f-d3eb-4e27-abeb-ec19c9e214cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:37.870747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a5c6f-d3eb-4e27-abeb-ec19c9e214cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:37.871507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a5c6f-d3eb-4e27-abeb-ec19c9e214cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:54:37.871819 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a5c6f-d3eb-4e27-abeb-ec19c9e214cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:54:37.872869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: WARNING py.warnings [None req-7f7a5c6f-d3eb-4e27-abeb-ec19c9e214cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:54:37.872869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: warnings.warn(msg) Jun 29 09:54:37.872869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: Jun 29 09:54:37.873777 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a5c6f-d3eb-4e27-abeb-ec19c9e214cb admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:54:37.886917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 743/1496] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:54:37.895334 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf586b-ded1-4853-a59a-a0c1507ca0fb None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:54:37.904954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf586b-ded1-4853-a59a-a0c1507ca0fb None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:54:37.909985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626fa3e-14dd-4c8f-b9b6-67516907128d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:54:37.910689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626fa3e-14dd-4c8f-b9b6-67516907128d admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:37.910975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626fa3e-14dd-4c8f-b9b6-67516907128d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:37.911256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626fa3e-14dd-4c8f-b9b6-67516907128d admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:37.911956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626fa3e-14dd-4c8f-b9b6-67516907128d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:54:37.913821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-f626fa3e-14dd-4c8f-b9b6-67516907128d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:54:37.913821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jun 29 09:54:37.913821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: Jun 29 09:54:37.915048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626fa3e-14dd-4c8f-b9b6-67516907128d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:54:37.954965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 754/1497] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:54:37 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:54:37.959430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a5c6f-d3eb-4e27-abeb-ec19c9e214cb admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:54:37.971021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a5c6f-d3eb-4e27-abeb-ec19c9e214cb admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:54:38.033262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b643e-59b4-4efc-8d69-0a92be016280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:54:38.033861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b643e-59b4-4efc-8d69-0a92be016280 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.034137 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b643e-59b4-4efc-8d69-0a92be016280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.034410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b643e-59b4-4efc-8d69-0a92be016280 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.035122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b643e-59b4-4efc-8d69-0a92be016280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:54:38.036142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: WARNING py.warnings [None req-9b2b643e-59b4-4efc-8d69-0a92be016280 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:54:38.036142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: warnings.warn(msg) Jun 29 09:54:38.036142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: Jun 29 09:54:38.037328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b643e-59b4-4efc-8d69-0a92be016280 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:54:38.058841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 744/1498] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:54:37 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:54:38.063321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626fa3e-14dd-4c8f-b9b6-67516907128d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:54:38.074771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626fa3e-14dd-4c8f-b9b6-67516907128d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:54:38.080179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc9db8-dafe-4d93-a80e-9be76e0c1e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:54:38.080912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc9db8-dafe-4d93-a80e-9be76e0c1e7b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.081197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc9db8-dafe-4d93-a80e-9be76e0c1e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.081473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc9db8-dafe-4d93-a80e-9be76e0c1e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.082219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc9db8-dafe-4d93-a80e-9be76e0c1e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:54:38.083297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-34cc9db8-dafe-4d93-a80e-9be76e0c1e7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:54:38.083297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jun 29 09:54:38.083297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: Jun 29 09:54:38.084457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc9db8-dafe-4d93-a80e-9be76e0c1e7b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:54:38.098909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 755/1499] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:54:38.103131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b643e-59b4-4efc-8d69-0a92be016280 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:54:38.114393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b643e-59b4-4efc-8d69-0a92be016280 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:54:38.120006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af9aab-10ca-4da9-b842-1e2aeef5aada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:54:38.120872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af9aab-10ca-4da9-b842-1e2aeef5aada admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.121157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af9aab-10ca-4da9-b842-1e2aeef5aada admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.121433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af9aab-10ca-4da9-b842-1e2aeef5aada admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.122139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af9aab-10ca-4da9-b842-1e2aeef5aada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:54:38.123212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: WARNING py.warnings [None req-74af9aab-10ca-4da9-b842-1e2aeef5aada admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:54:38.123212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: warnings.warn(msg) Jun 29 09:54:38.123212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: Jun 29 09:54:38.124126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af9aab-10ca-4da9-b842-1e2aeef5aada admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:54:38.139824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 745/1500] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:54:38.144437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc9db8-dafe-4d93-a80e-9be76e0c1e7b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:54:38.156804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc9db8-dafe-4d93-a80e-9be76e0c1e7b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:54:38.163156 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045f46e-4b6a-4ed4-be2e-efaa2103e28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:54:38.164126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045f46e-4b6a-4ed4-be2e-efaa2103e28e admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.164418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045f46e-4b6a-4ed4-be2e-efaa2103e28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.164696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045f46e-4b6a-4ed4-be2e-efaa2103e28e admin admin] PATH_INFO: `/v3/projects/40e3b125ab6b4842b9501abc34438ce0/users/47ab337036eb4e47a30140116b3e1f7e/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.181828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045f46e-4b6a-4ed4-be2e-efaa2103e28e admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e3b125ab6b4842b9501abc34438ce0, user_id=47ab337036eb4e47a30140116b3e1f7e, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:54:38.182933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-c045f46e-4b6a-4ed4-be2e-efaa2103e28e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:54:38.182933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jun 29 09:54:38.182933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: Jun 29 09:54:38.184884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045f46e-4b6a-4ed4-be2e-efaa2103e28e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:54:38.203124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 756/1501] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:54:38 2022] PUT /identity/v3/projects/40e3b125ab6b4842b9501abc34438ce0/users/47ab337036eb4e47a30140116b3e1f7e/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:54:38.215656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5219561-2452-4dd9-80d3-bc35302cbabd None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.215924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5219561-2452-4dd9-80d3-bc35302cbabd None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.216209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5219561-2452-4dd9-80d3-bc35302cbabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.245523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-e5219561-2452-4dd9-80d3-bc35302cbabd None None] MFA Rules not processed for user `47ab337036eb4e47a30140116b3e1f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:54:38.246697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e5219561-2452-4dd9-80d3-bc35302cbabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:54:38.318361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 746/1502] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:54:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:54:38.332473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c2469-693c-44eb-a897-d9e8da244b4f None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.332756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c2469-693c-44eb-a897-d9e8da244b4f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.333135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c2469-693c-44eb-a897-d9e8da244b4f None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.334445 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 757/1503] 10.222.0.52 () {54 vars in 963 bytes} [Wed Jun 29 09:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:54:38.339891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f87a59-f2a7-446d-ad26-d11fdff51cdd None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.340149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f87a59-f2a7-446d-ad26-d11fdff51cdd None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.340461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f87a59-f2a7-446d-ad26-d11fdff51cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.360028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-29f87a59-f2a7-446d-ad26-d11fdff51cdd None None] MFA Rules not processed for user `34813be7e8fd4fcfb5907bee6a465d16`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:54:38.361290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-29f87a59-f2a7-446d-ad26-d11fdff51cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:54:38.408826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:54:38.417625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 747/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Jun 29 09:54:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:54:38.425346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c2469-693c-44eb-a897-d9e8da244b4f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:54:38.437365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c2469-693c-44eb-a897-d9e8da244b4f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:54:38.441508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2e843-c9eb-46b6-a7bb-c109cf20d1a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:54:38.442171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e843-c9eb-46b6-a7bb-c109cf20d1a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.442450 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e843-c9eb-46b6-a7bb-c109cf20d1a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.442762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e843-c9eb-46b6-a7bb-c109cf20d1a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.452098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2e843-c9eb-46b6-a7bb-c109cf20d1a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:54:38.453549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2e843-c9eb-46b6-a7bb-c109cf20d1a5 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:54:38.462846 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 758/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:54:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:54:38.509938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deedcf1-b978-4982-8f15-bfe539b83af7 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.510234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deedcf1-b978-4982-8f15-bfe539b83af7 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.510512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deedcf1-b978-4982-8f15-bfe539b83af7 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.512025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 748/1506] 10.222.0.52 () {52 vars in 829 bytes} [Wed Jun 29 09:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:54:38.516035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2e843-c9eb-46b6-a7bb-c109cf20d1a5 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:54:38.581448 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2e843-c9eb-46b6-a7bb-c109cf20d1a5 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:54:38.586555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:54:38.587473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.587765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.588039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] PATH_INFO: `/v3/users/47ab337036eb4e47a30140116b3e1f7e` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.589411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ab337036eb4e47a30140116b3e1f7e)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:54:38.590434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:54:38.590434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jun 29 09:54:38.590434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: Jun 29 09:54:38.592063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:54:38.666817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ab337036eb4e47a30140116b3e1f7e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:54:38.694975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ab337036eb4e47a30140116b3e1f7e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:54:38.695978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ab337036eb4e47a30140116b3e1f7e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:54:38.713253 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ab337036eb4e47a30140116b3e1f7e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:54:38.714178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ab337036eb4e47a30140116b3e1f7e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:54:38.719204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:54:38.733880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 759/1507] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:54:38 2022] DELETE /identity/v3/users/47ab337036eb4e47a30140116b3e1f7e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:54:38.739540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e12aa-56c5-479f-b8fa-5b197836e1db None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.739990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e12aa-56c5-479f-b8fa-5b197836e1db None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.740440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e12aa-56c5-479f-b8fa-5b197836e1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.766693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-036e12aa-56c5-479f-b8fa-5b197836e1db None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:54:38.768307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-036e12aa-56c5-479f-b8fa-5b197836e1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:54:38.842790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 749/1508] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:54:38.853640 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:54:38.855352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:54:38.921700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383678b9-a1e3-44f4-a5b7-ecf00fa7730f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:54:38.926733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284754e-2c01-45f8-b70d-a2014a9e649b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:54:38.927419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284754e-2c01-45f8-b70d-a2014a9e649b service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:38.927703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284754e-2c01-45f8-b70d-a2014a9e649b service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:38.927976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284754e-2c01-45f8-b70d-a2014a9e649b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:38.938597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284754e-2c01-45f8-b70d-a2014a9e649b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:54:38.940077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284754e-2c01-45f8-b70d-a2014a9e649b service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:54:38.950984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 760/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:54:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:54:39.346177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e12aa-56c5-479f-b8fa-5b197836e1db None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:54:39.347928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-036e12aa-56c5-479f-b8fa-5b197836e1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:54:39.360928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e12aa-56c5-479f-b8fa-5b197836e1db None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:54:39.365575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:54:39.366346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:54:39.366736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:54:39.366892 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] PATH_INFO: `/v3/projects/40e3b125ab6b4842b9501abc34438ce0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:54:39.368146 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40e3b125ab6b4842b9501abc34438ce0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:54:39.369111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: WARNING py.warnings [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:54:39.369111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: warnings.warn(msg) Jun 29 09:54:39.369111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: Jun 29 09:54:39.370854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:54:39.450875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40e3b125ab6b4842b9501abc34438ce0', 'request_id': 'req-d1026c26-d87a-4271-aeed-e30ff95065f9'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:54:39.460310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40e3b125ab6b4842b9501abc34438ce0', 'request_id': 'req-d1026c26-d87a-4271-aeed-e30ff95065f9'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:54:39.467150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40e3b125ab6b4842b9501abc34438ce0', 'request_id': 'req-d1026c26-d87a-4271-aeed-e30ff95065f9'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:54:39.486407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] The token cache is being invalidate because project 40e3b125ab6b4842b9501abc34438ce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:54:39.486719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:54:39.487581 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d1026c26-d87a-4271-aeed-e30ff95065f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:54:39.489275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 750/1510] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:54:39 2022] DELETE /identity/v3/projects/40e3b125ab6b4842b9501abc34438ce0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:54:48.946047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:54:49.483134 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:56:20.020553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec889171-8cb5-482a-a9e0-e982c2730cb3 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:20.021417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec889171-8cb5-482a-a9e0-e982c2730cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:20.021417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec889171-8cb5-482a-a9e0-e982c2730cb3 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:20.022891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:56:20.028508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997fd96-de05-40c5-ba39-2043f12a5321 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:20.028997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997fd96-de05-40c5-ba39-2043f12a5321 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:20.029098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997fd96-de05-40c5-ba39-2043f12a5321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:20.049063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-f997fd96-de05-40c5-ba39-2043f12a5321 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:56:20.050302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f997fd96-de05-40c5-ba39-2043f12a5321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:20.121680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:56:20.278857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec889171-8cb5-482a-a9e0-e982c2730cb3 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:56:20.281064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ec889171-8cb5-482a-a9e0-e982c2730cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:20.358742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec889171-8cb5-482a-a9e0-e982c2730cb3 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:56:20.364873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2621a-0306-4dcc-b409-0c61010f4cb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:56:20.365555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2621a-0306-4dcc-b409-0c61010f4cb1 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:20.365835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2621a-0306-4dcc-b409-0c61010f4cb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:20.366107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2621a-0306-4dcc-b409-0c61010f4cb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:20.374925 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2621a-0306-4dcc-b409-0c61010f4cb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:56:20.376557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2621a-0306-4dcc-b409-0c61010f4cb1 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:56:20.386383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 762/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:56:21.233423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49349f7a-8fac-43e6-a800-c8db3f7548c1 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:21.234072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49349f7a-8fac-43e6-a800-c8db3f7548c1 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:21.234072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49349f7a-8fac-43e6-a800-c8db3f7548c1 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:21.235419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:56:21.240453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c8bc-7802-4f04-8bde-60165f3cb50a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:21.240899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c8bc-7802-4f04-8bde-60165f3cb50a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:21.241007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c8bc-7802-4f04-8bde-60165f3cb50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:21.260643 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9bf3c8bc-7802-4f04-8bde-60165f3cb50a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:56:21.261859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9bf3c8bc-7802-4f04-8bde-60165f3cb50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:21.271098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:56:21.424090 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49349f7a-8fac-43e6-a800-c8db3f7548c1 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:56:21.440193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49349f7a-8fac-43e6-a800-c8db3f7548c1 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:56:21.445938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a9830-f7d5-4b9f-8dc8-a8a4ba1adcd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:56:21.446614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a9830-f7d5-4b9f-8dc8-a8a4ba1adcd3 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:21.446921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a9830-f7d5-4b9f-8dc8-a8a4ba1adcd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:21.447199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a9830-f7d5-4b9f-8dc8-a8a4ba1adcd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:21.456600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a9830-f7d5-4b9f-8dc8-a8a4ba1adcd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:56:21.458351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a9830-f7d5-4b9f-8dc8-a8a4ba1adcd3 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:56:21.470023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 753/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:56:21.931493 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26fbfc-4c09-4886-9967-741dde4630c2 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:21.931882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26fbfc-4c09-4886-9967-741dde4630c2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:21.932138 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26fbfc-4c09-4886-9967-741dde4630c2 None None] PATH_INFO: `/v3` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:21.933455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 764/1517] 10.222.0.58 () {54 vars in 925 bytes} [Wed Jun 29 09:56:21 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:56:21.956690 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624be95a-1ec7-4015-a4a0-887b99a4c86b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:21.956951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624be95a-1ec7-4015-a4a0-887b99a4c86b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:21.957209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624be95a-1ec7-4015-a4a0-887b99a4c86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:21.977887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-624be95a-1ec7-4015-a4a0-887b99a4c86b None None] MFA Rules not processed for user `e9bff6c855ea46feaa5526d2c27c9026`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:56:21.979183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-624be95a-1ec7-4015-a4a0-887b99a4c86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:22.047450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 754/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Jun 29 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:56:25.797764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71e2dd-9a5e-4f71-9a42-1dcecd095d38 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:25.798316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71e2dd-9a5e-4f71-9a42-1dcecd095d38 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:25.798316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71e2dd-9a5e-4f71-9a42-1dcecd095d38 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:25.799671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 765/1519] 10.222.0.52 () {54 vars in 960 bytes} [Wed Jun 29 09:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:56:25.805130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492928-ba53-4610-ae7a-03a5164ca3b6 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:25.805600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492928-ba53-4610-ae7a-03a5164ca3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:25.805696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492928-ba53-4610-ae7a-03a5164ca3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:25.824879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fa492928-ba53-4610-ae7a-03a5164ca3b6 None None] MFA Rules not processed for user `e9bff6c855ea46feaa5526d2c27c9026`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:56:25.826195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fa492928-ba53-4610-ae7a-03a5164ca3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:25.835046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 755/1520] 10.222.0.52 () {58 vars in 1057 bytes} [Wed Jun 29 09:56:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:56:25.844047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71e2dd-9a5e-4f71-9a42-1dcecd095d38 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:56:25.859948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71e2dd-9a5e-4f71-9a42-1dcecd095d38 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:56:25.865649 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cf1c5-e269-4816-bfa7-8cf1b4f08735 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:56:25.866304 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cf1c5-e269-4816-bfa7-8cf1b4f08735 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:25.866604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cf1c5-e269-4816-bfa7-8cf1b4f08735 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:25.866912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cf1c5-e269-4816-bfa7-8cf1b4f08735 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:25.876397 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3cf1c5-e269-4816-bfa7-8cf1b4f08735 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:56:25.878024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3cf1c5-e269-4816-bfa7-8cf1b4f08735 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:56:25.888356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 766/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:56:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:56:32.710655 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf659458-6cc5-4d47-b4bd-b1d66c10df04 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:32.711272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf659458-6cc5-4d47-b4bd-b1d66c10df04 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:32.711272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf659458-6cc5-4d47-b4bd-b1d66c10df04 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:32.712627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:56:32.719638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dda49-fabe-4bfb-a027-f9833564a352 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:32.720127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dda49-fabe-4bfb-a027-f9833564a352 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:32.720238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dda49-fabe-4bfb-a027-f9833564a352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:32.741470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0d4dda49-fabe-4bfb-a027-f9833564a352 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:56:32.742729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0d4dda49-fabe-4bfb-a027-f9833564a352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:32.762745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:56:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:56:32.916887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf659458-6cc5-4d47-b4bd-b1d66c10df04 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:56:32.933421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf659458-6cc5-4d47-b4bd-b1d66c10df04 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:56:32.939375 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933c570-238d-4032-95e2-de9f4712e0a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:56:32.940025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933c570-238d-4032-95e2-de9f4712e0a4 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:32.940307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933c570-238d-4032-95e2-de9f4712e0a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:32.940583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933c570-238d-4032-95e2-de9f4712e0a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:32.949379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933c570-238d-4032-95e2-de9f4712e0a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:56:32.951032 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933c570-238d-4032-95e2-de9f4712e0a4 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:56:32.961373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 757/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:56:33.792667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19307218-214d-445a-9f84-b36c27357bf0 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:33.793510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19307218-214d-445a-9f84-b36c27357bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:33.793510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19307218-214d-445a-9f84-b36c27357bf0 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:33.795713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 09:56:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:56:33.801464 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ce931-15f9-462e-b793-633bfcfcc00f None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:33.801957 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ce931-15f9-462e-b793-633bfcfcc00f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:33.801957 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ce931-15f9-462e-b793-633bfcfcc00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:33.824082 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b68ce931-15f9-462e-b793-633bfcfcc00f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:56:33.825503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b68ce931-15f9-462e-b793-633bfcfcc00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:33.834771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 09:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:56:33.993834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19307218-214d-445a-9f84-b36c27357bf0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:56:34.010269 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19307218-214d-445a-9f84-b36c27357bf0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:56:34.016906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091e5ab-1ead-4fd5-8dbc-fe97c6c820cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:56:34.017603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091e5ab-1ead-4fd5-8dbc-fe97c6c820cb service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:34.017884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091e5ab-1ead-4fd5-8dbc-fe97c6c820cb service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:34.018161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091e5ab-1ead-4fd5-8dbc-fe97c6c820cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:34.027310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091e5ab-1ead-4fd5-8dbc-fe97c6c820cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:56:34.028923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091e5ab-1ead-4fd5-8dbc-fe97c6c820cb service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:56:34.039190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 769/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:56:37.971691 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633da80-4c4a-4eb3-beeb-b542d73e1c00 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:37.972310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633da80-4c4a-4eb3-beeb-b542d73e1c00 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:37.972310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633da80-4c4a-4eb3-beeb-b542d73e1c00 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:37.973646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:56:37.979303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e9e30-8ace-477f-ae6e-9654f1f58383 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:37.979794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e9e30-8ace-477f-ae6e-9654f1f58383 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:37.979900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e9e30-8ace-477f-ae6e-9654f1f58383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:38.000066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-392e9e30-8ace-477f-ae6e-9654f1f58383 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:56:38.001286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-392e9e30-8ace-477f-ae6e-9654f1f58383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:38.009976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:56:38.052395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633da80-4c4a-4eb3-beeb-b542d73e1c00 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:56:38.054504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b633da80-4c4a-4eb3-beeb-b542d73e1c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:38.115879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633da80-4c4a-4eb3-beeb-b542d73e1c00 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:56:38.119831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e878b-fa8a-43ce-9a1d-65576ead14ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:56:38.120492 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e878b-fa8a-43ce-9a1d-65576ead14ef service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:38.120772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e878b-fa8a-43ce-9a1d-65576ead14ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:38.121065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e878b-fa8a-43ce-9a1d-65576ead14ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:38.131355 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e878b-fa8a-43ce-9a1d-65576ead14ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:56:38.132818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e878b-fa8a-43ce-9a1d-65576ead14ef service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:56:38.138039 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:56:38.143591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 760/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:56:40.191694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dce3d-a044-49ce-bd07-8544890e165d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:40.192277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dce3d-a044-49ce-bd07-8544890e165d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:40.192277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dce3d-a044-49ce-bd07-8544890e165d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:40.193710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:56:40.199172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbdeb5-ecc8-46a4-8558-25d389bfd81e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:40.199675 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbdeb5-ecc8-46a4-8558-25d389bfd81e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:40.199769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbdeb5-ecc8-46a4-8558-25d389bfd81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:40.220434 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-d1fbdeb5-ecc8-46a4-8558-25d389bfd81e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:56:40.221568 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d1fbdeb5-ecc8-46a4-8558-25d389bfd81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:40.229545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:56:40.266120 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dce3d-a044-49ce-bd07-8544890e165d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:56:40.278501 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dce3d-a044-49ce-bd07-8544890e165d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:56:40.282872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5189404-0273-4e5b-a070-93a5bdec9cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:56:40.283523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5189404-0273-4e5b-a070-93a5bdec9cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:40.283801 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5189404-0273-4e5b-a070-93a5bdec9cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:40.284074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5189404-0273-4e5b-a070-93a5bdec9cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:40.293705 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5189404-0273-4e5b-a070-93a5bdec9cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:56:40.295196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5189404-0273-4e5b-a070-93a5bdec9cc8 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:56:40.305001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 772/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:56:43.854258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afb17a-0403-4b7e-a6b6-fb74c501d654 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:43.854981 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afb17a-0403-4b7e-a6b6-fb74c501d654 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:43.854981 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afb17a-0403-4b7e-a6b6-fb74c501d654 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:43.856222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:56:43.862638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bbe89-ffab-40b2-b1ef-726cbe673ca7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:43.863203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bbe89-ffab-40b2-b1ef-726cbe673ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:43.863203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bbe89-ffab-40b2-b1ef-726cbe673ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:43.884081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ee8bbe89-ffab-40b2-b1ef-726cbe673ca7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:56:43.885386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ee8bbe89-ffab-40b2-b1ef-726cbe673ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:43.894491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:56:44.205193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afb17a-0403-4b7e-a6b6-fb74c501d654 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:56:44.207060 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f9afb17a-0403-4b7e-a6b6-fb74c501d654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:44.267869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afb17a-0403-4b7e-a6b6-fb74c501d654 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:56:44.271916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4aede7-895e-4e2c-a055-9896e8e84267 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:56:44.272571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aede7-895e-4e2c-a055-9896e8e84267 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:44.272850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aede7-895e-4e2c-a055-9896e8e84267 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:44.273124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aede7-895e-4e2c-a055-9896e8e84267 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:44.283204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4aede7-895e-4e2c-a055-9896e8e84267 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:56:44.284650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4aede7-895e-4e2c-a055-9896e8e84267 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:56:44.294501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 763/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:56:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:56:46.375659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2175f-09d8-4f62-ab7b-283fbb0777c8 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:46.376221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2175f-09d8-4f62-ab7b-283fbb0777c8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:46.376221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2175f-09d8-4f62-ab7b-283fbb0777c8 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:46.377528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 09:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:56:46.383251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472c9c9-023e-4401-ae1d-116bfff25a9b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:46.383763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472c9c9-023e-4401-ae1d-116bfff25a9b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:46.383763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472c9c9-023e-4401-ae1d-116bfff25a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:46.403249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-9472c9c9-023e-4401-ae1d-116bfff25a9b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:56:46.404470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9472c9c9-023e-4401-ae1d-116bfff25a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:46.413452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 09:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:56:46.729652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2175f-09d8-4f62-ab7b-283fbb0777c8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:56:46.731471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-abf2175f-09d8-4f62-ab7b-283fbb0777c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:56:46.757694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2175f-09d8-4f62-ab7b-283fbb0777c8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:56:46.761427 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c526a4-f8ba-4f02-8eaf-6a1cac999daa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:56:46.764455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c526a4-f8ba-4f02-8eaf-6a1cac999daa service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:56:46.764455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c526a4-f8ba-4f02-8eaf-6a1cac999daa service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:56:46.764455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c526a4-f8ba-4f02-8eaf-6a1cac999daa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:56:46.780771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c526a4-f8ba-4f02-8eaf-6a1cac999daa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:56:46.782228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c526a4-f8ba-4f02-8eaf-6a1cac999daa service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:56:46.792189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 775/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:56:56.397659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=32 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:56:56.787798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=70 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:14.766904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c599d72-c354-418f-9bcb-1518ad36cb4d None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:14.767572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c599d72-c354-418f-9bcb-1518ad36cb4d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:14.767572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c599d72-c354-418f-9bcb-1518ad36cb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:14.795957 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-1c599d72-c354-418f-9bcb-1518ad36cb4d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:14.797095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-1c599d72-c354-418f-9bcb-1518ad36cb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:14.812240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 765/1540] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:14.821657 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37031afb-651e-4cff-8ac0-c8bb1a00bf03 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:14.823027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37031afb-651e-4cff-8ac0-c8bb1a00bf03 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:14.823525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37031afb-651e-4cff-8ac0-c8bb1a00bf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:14.848819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-37031afb-651e-4cff-8ac0-c8bb1a00bf03 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:14.850517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-37031afb-651e-4cff-8ac0-c8bb1a00bf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:14.863374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 776/1541] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:58:14.868821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c599d72-c354-418f-9bcb-1518ad36cb4d None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:14.882517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c599d72-c354-418f-9bcb-1518ad36cb4d None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:14.891152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2110288-bfac-4eeb-8f4b-2def162e0b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:14.891880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2110288-bfac-4eeb-8f4b-2def162e0b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:14.892101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2110288-bfac-4eeb-8f4b-2def162e0b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:14.892371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2110288-bfac-4eeb-8f4b-2def162e0b6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:14.893059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebacf5-611b-4d84-9615-1e962e09f7e5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:14.893346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2110288-bfac-4eeb-8f4b-2def162e0b6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:14.893676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2110288-bfac-4eeb-8f4b-2def162e0b6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:14.894291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebacf5-611b-4d84-9615-1e962e09f7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:14.899018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2110288-bfac-4eeb-8f4b-2def162e0b6b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:14.899967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebacf5-611b-4d84-9615-1e962e09f7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:14.921260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 766/1542] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:14.928689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-afebacf5-611b-4d84-9615-1e962e09f7e5 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:14.930755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-afebacf5-611b-4d84-9615-1e962e09f7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:14.938984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2110288-bfac-4eeb-8f4b-2def162e0b6b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:14.946006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 777/1543] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:58:14.958590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2110288-bfac-4eeb-8f4b-2def162e0b6b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:14.962312 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68f6c2-f800-4693-a551-fe8ba10d9625 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:14.963336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68f6c2-f800-4693-a551-fe8ba10d9625 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:14.963832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68f6c2-f800-4693-a551-fe8ba10d9625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:14.982591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ff260-63d8-449b-a259-ec0c2ea7be2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:14.982591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff260-63d8-449b-a259-ec0c2ea7be2b admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:14.982591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff260-63d8-449b-a259-ec0c2ea7be2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:14.982591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff260-63d8-449b-a259-ec0c2ea7be2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:14.983077 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ff260-63d8-449b-a259-ec0c2ea7be2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:14.984139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: WARNING py.warnings [None req-007ff260-63d8-449b-a259-ec0c2ea7be2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:58:14.984139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: warnings.warn(msg) Jun 29 09:58:14.984139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: Jun 29 09:58:14.985867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ff260-63d8-449b-a259-ec0c2ea7be2b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:14.991913 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1b68f6c2-f800-4693-a551-fe8ba10d9625 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:14.993481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1b68f6c2-f800-4693-a551-fe8ba10d9625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.015367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 778/1544] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.022111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68f6c2-f800-4693-a551-fe8ba10d9625 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.043861 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68f6c2-f800-4693-a551-fe8ba10d9625 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.065685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 767/1545] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:14 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:15.067445 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0a66e-7fc4-47c3-afd4-2efeb9966d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.068558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0a66e-7fc4-47c3-afd4-2efeb9966d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.069078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0a66e-7fc4-47c3-afd4-2efeb9966d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.069577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0a66e-7fc4-47c3-afd4-2efeb9966d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.070534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0a66e-7fc4-47c3-afd4-2efeb9966d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:15.071069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0a66e-7fc4-47c3-afd4-2efeb9966d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.072339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-b9b0a66e-7fc4-47c3-afd4-2efeb9966d6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:58:15.072339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jun 29 09:58:15.072339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: Jun 29 09:58:15.073665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0a66e-7fc4-47c3-afd4-2efeb9966d6b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.079679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ff260-63d8-449b-a259-ec0c2ea7be2b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.098298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 779/1546] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:15.100199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d565c4b-c2d9-476c-a9dd-70877359feb0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.101180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d565c4b-c2d9-476c-a9dd-70877359feb0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.101683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d565c4b-c2d9-476c-a9dd-70877359feb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.138595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9d565c4b-c2d9-476c-a9dd-70877359feb0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.140572 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9d565c4b-c2d9-476c-a9dd-70877359feb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.214047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ff260-63d8-449b-a259-ec0c2ea7be2b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.218511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c93afe-0f28-4b6d-97bf-8d25a153b53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.224590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c93afe-0f28-4b6d-97bf-8d25a153b53f admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.224590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c93afe-0f28-4b6d-97bf-8d25a153b53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.224590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c93afe-0f28-4b6d-97bf-8d25a153b53f admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.224590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c93afe-0f28-4b6d-97bf-8d25a153b53f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.224590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c93afe-0f28-4b6d-97bf-8d25a153b53f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.227834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 780/1547] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.228865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d565c4b-c2d9-476c-a9dd-70877359feb0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.239155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d565c4b-c2d9-476c-a9dd-70877359feb0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.244046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfd186-7bf3-473f-bead-f80a808519ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.244720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfd186-7bf3-473f-bead-f80a808519ab admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.244997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfd186-7bf3-473f-bead-f80a808519ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.245267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfd186-7bf3-473f-bead-f80a808519ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.245966 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfd186-7bf3-473f-bead-f80a808519ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.246391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 768/1548] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:15.247864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753d5eb-e10f-42f6-9604-01abc1bc5243 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.248157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753d5eb-e10f-42f6-9604-01abc1bc5243 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.248281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfd186-7bf3-473f-bead-f80a808519ab admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.248430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753d5eb-e10f-42f6-9604-01abc1bc5243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.269173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-1753d5eb-e10f-42f6-9604-01abc1bc5243 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.270380 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-1753d5eb-e10f-42f6-9604-01abc1bc5243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.277982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 781/1549] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:15.278450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 769/1550] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.279432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf3104-ea9f-47c2-b8b6-c88fa692990c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.279723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf3104-ea9f-47c2-b8b6-c88fa692990c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.279833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a3afe-7a7c-40ed-92a0-00694dd7bb93 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.280004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf3104-ea9f-47c2-b8b6-c88fa692990c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.280109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a3afe-7a7c-40ed-92a0-00694dd7bb93 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.280272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a3afe-7a7c-40ed-92a0-00694dd7bb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.307092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b65a3afe-7a7c-40ed-92a0-00694dd7bb93 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.308566 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b65a3afe-7a7c-40ed-92a0-00694dd7bb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.309576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-75bf3104-ea9f-47c2-b8b6-c88fa692990c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.310855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-75bf3104-ea9f-47c2-b8b6-c88fa692990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.373805 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 770/1551] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.376686 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a916d-a40d-44e8-b30d-c1e208770b56 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.376686 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a916d-a40d-44e8-b30d-c1e208770b56 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.376686 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a916d-a40d-44e8-b30d-c1e208770b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.392745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 782/1552] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.394251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080cdad-6915-4ff8-9938-a7dab94b452d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.394545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080cdad-6915-4ff8-9938-a7dab94b452d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.394851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080cdad-6915-4ff8-9938-a7dab94b452d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.399838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fa6a916d-a40d-44e8-b30d-c1e208770b56 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.401542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fa6a916d-a40d-44e8-b30d-c1e208770b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.410208 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 771/1553] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.411063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a916d-a40d-44e8-b30d-c1e208770b56 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.413403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4080cdad-6915-4ff8-9938-a7dab94b452d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.414642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4080cdad-6915-4ff8-9938-a7dab94b452d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.422625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a916d-a40d-44e8-b30d-c1e208770b56 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.423069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 783/1554] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.423812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080cdad-6915-4ff8-9938-a7dab94b452d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.427326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c2dab-2cd8-4668-9701-66c756da18ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.428008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c2dab-2cd8-4668-9701-66c756da18ba admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.428288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c2dab-2cd8-4668-9701-66c756da18ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.428570 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c2dab-2cd8-4668-9701-66c756da18ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.429296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c2dab-2cd8-4668-9701-66c756da18ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.430947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c2dab-2cd8-4668-9701-66c756da18ba admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.433834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080cdad-6915-4ff8-9938-a7dab94b452d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.438361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636a656-b2d5-4777-add7-ea4dc8eec525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.439067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636a656-b2d5-4777-add7-ea4dc8eec525 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.439354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636a656-b2d5-4777-add7-ea4dc8eec525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.439691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636a656-b2d5-4777-add7-ea4dc8eec525 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.440845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636a656-b2d5-4777-add7-ea4dc8eec525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.441373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 772/1555] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:15.441879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-3636a656-b2d5-4777-add7-ea4dc8eec525 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:58:15.441879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jun 29 09:58:15.441879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: Jun 29 09:58:15.443225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636a656-b2d5-4777-add7-ea4dc8eec525 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.443344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc920834-bd51-4825-88c4-9a0c6e777cdb None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.443605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc920834-bd51-4825-88c4-9a0c6e777cdb None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.443886 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc920834-bd51-4825-88c4-9a0c6e777cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.462452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-bc920834-bd51-4825-88c4-9a0c6e777cdb None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.464117 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-bc920834-bd51-4825-88c4-9a0c6e777cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.466788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 784/1556] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:15.468134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49a93-54ea-4acc-8331-96ee067d35fa None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.468427 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49a93-54ea-4acc-8331-96ee067d35fa None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.468702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49a93-54ea-4acc-8331-96ee067d35fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.472958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 773/1557] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.474084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7c86-ccb7-4f21-801b-26b6e74d1571 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.474371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7c86-ccb7-4f21-801b-26b6e74d1571 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.474670 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7c86-ccb7-4f21-801b-26b6e74d1571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.489800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0fb49a93-54ea-4acc-8331-96ee067d35fa None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.491174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0fb49a93-54ea-4acc-8331-96ee067d35fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.492516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-61eb7c86-ccb7-4f21-801b-26b6e74d1571 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.493674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-61eb7c86-ccb7-4f21-801b-26b6e74d1571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.499257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 785/1558] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.500572 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0e2b-9067-4548-9498-946906f620cd None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.500946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0e2b-9067-4548-9498-946906f620cd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.501290 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0e2b-9067-4548-9498-946906f620cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.506733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 774/1559] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.506733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8f3b1-e9e3-46f6-acf9-63de70528167 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.506733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8f3b1-e9e3-46f6-acf9-63de70528167 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.506733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8f3b1-e9e3-46f6-acf9-63de70528167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.524237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-28e8f3b1-e9e3-46f6-acf9-63de70528167 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.525060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-db8c0e2b-9067-4548-9498-946906f620cd None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.525598 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-28e8f3b1-e9e3-46f6-acf9-63de70528167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.526222 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-db8c0e2b-9067-4548-9498-946906f620cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.533705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 775/1560] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.537640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8f3b1-e9e3-46f6-acf9-63de70528167 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.537744 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 786/1561] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.537744 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c0e2b-9067-4548-9498-946906f620cd None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.544107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:15.545573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c0e2b-9067-4548-9498-946906f620cd None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.548457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8f3b1-e9e3-46f6-acf9-63de70528167 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.550664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfdacf-3ed6-43b2-878d-11f3ee87b786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.551345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfdacf-3ed6-43b2-878d-11f3ee87b786 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.551652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfdacf-3ed6-43b2-878d-11f3ee87b786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.551927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfdacf-3ed6-43b2-878d-11f3ee87b786 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.552837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cfdacf-3ed6-43b2-878d-11f3ee87b786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.554458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cfdacf-3ed6-43b2-878d-11f3ee87b786 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.554729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1751ac4-839c-418c-9798-28fdb48e453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.555405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1751ac4-839c-418c-9798-28fdb48e453e admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.555686 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1751ac4-839c-418c-9798-28fdb48e453e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.555954 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1751ac4-839c-418c-9798-28fdb48e453e admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.556688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1751ac4-839c-418c-9798-28fdb48e453e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:15.556830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:15.557001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1751ac4-839c-418c-9798-28fdb48e453e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.558792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1751ac4-839c-418c-9798-28fdb48e453e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.567532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 787/1562] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:15.568889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfdacf-3ed6-43b2-878d-11f3ee87b786 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.574272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 776/1563] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:15.575185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1751ac4-839c-418c-9798-28fdb48e453e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.580451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfdacf-3ed6-43b2-878d-11f3ee87b786 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.584993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14b933-2320-492d-8093-7228b87a92f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.585650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14b933-2320-492d-8093-7228b87a92f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.585931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14b933-2320-492d-8093-7228b87a92f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.586207 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14b933-2320-492d-8093-7228b87a92f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.587256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14b933-2320-492d-8093-7228b87a92f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.589849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14b933-2320-492d-8093-7228b87a92f5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.592330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1751ac4-839c-418c-9798-28fdb48e453e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.598857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86551182-9d61-4d51-bee6-95d5c3580e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.599526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86551182-9d61-4d51-bee6-95d5c3580e8f admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.599808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86551182-9d61-4d51-bee6-95d5c3580e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.600075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86551182-9d61-4d51-bee6-95d5c3580e8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.600476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 788/1564] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:15.600799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86551182-9d61-4d51-bee6-95d5c3580e8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:15.601102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86551182-9d61-4d51-bee6-95d5c3580e8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.601291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14b933-2320-492d-8093-7228b87a92f5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.602815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86551182-9d61-4d51-bee6-95d5c3580e8f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.610869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14b933-2320-492d-8093-7228b87a92f5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.615891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3d7a5-2f9c-46f3-aa6b-061177f1f156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.616556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3d7a5-2f9c-46f3-aa6b-061177f1f156 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.616807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3d7a5-2f9c-46f3-aa6b-061177f1f156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.617079 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3d7a5-2f9c-46f3-aa6b-061177f1f156 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.617805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3d7a5-2f9c-46f3-aa6b-061177f1f156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:15.618107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3d7a5-2f9c-46f3-aa6b-061177f1f156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.618211 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 777/1565] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:15.619515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3166035-38e7-4d39-9d4b-43d12501784e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.619745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3d7a5-2f9c-46f3-aa6b-061177f1f156 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.619863 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3166035-38e7-4d39-9d4b-43d12501784e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.620066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3166035-38e7-4d39-9d4b-43d12501784e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.640197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-f3166035-38e7-4d39-9d4b-43d12501784e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:15.640598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 789/1566] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:15.641405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f3166035-38e7-4d39-9d4b-43d12501784e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:15.641553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3d7a5-2f9c-46f3-aa6b-061177f1f156 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.649825 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 778/1567] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:15.650549 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3166035-38e7-4d39-9d4b-43d12501784e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.653016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3d7a5-2f9c-46f3-aa6b-061177f1f156 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.657610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8752f-773e-4fce-bb4c-1606a83e1ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.658276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8752f-773e-4fce-bb4c-1606a83e1ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.658621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8752f-773e-4fce-bb4c-1606a83e1ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.659804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8752f-773e-4fce-bb4c-1606a83e1ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.660193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3166035-38e7-4d39-9d4b-43d12501784e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.660539 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8752f-773e-4fce-bb4c-1606a83e1ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:15.660848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8752f-773e-4fce-bb4c-1606a83e1ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.662456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8752f-773e-4fce-bb4c-1606a83e1ee3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.664705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350b2bb-f2cd-4d9d-98dc-5047e4e875df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.665370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350b2bb-f2cd-4d9d-98dc-5047e4e875df admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.665651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350b2bb-f2cd-4d9d-98dc-5047e4e875df admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.665922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350b2bb-f2cd-4d9d-98dc-5047e4e875df admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.666719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350b2bb-f2cd-4d9d-98dc-5047e4e875df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:15.667101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350b2bb-f2cd-4d9d-98dc-5047e4e875df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.668704 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350b2bb-f2cd-4d9d-98dc-5047e4e875df admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.675707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 790/1568] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:15.676542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8752f-773e-4fce-bb4c-1606a83e1ee3 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.683527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 779/1569] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:15.684400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350b2bb-f2cd-4d9d-98dc-5047e4e875df admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.686226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8752f-773e-4fce-bb4c-1606a83e1ee3 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.691451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ba7cf-e33f-49c7-a761-51c20e41b9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.692354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ba7cf-e33f-49c7-a761-51c20e41b9a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.692647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ba7cf-e33f-49c7-a761-51c20e41b9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.692931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ba7cf-e33f-49c7-a761-51c20e41b9a0 admin admin] PATH_INFO: `/v3/projects/c1f89948b1e9480583555977ac15a6e0/users/54a1953b9908435592e6b38e7de9435c/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.695230 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350b2bb-f2cd-4d9d-98dc-5047e4e875df admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.699937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded11809-e96a-475d-ad0b-b2790512795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.700593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded11809-e96a-475d-ad0b-b2790512795e admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.700898 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded11809-e96a-475d-ad0b-b2790512795e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.701167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded11809-e96a-475d-ad0b-b2790512795e admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.701860 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded11809-e96a-475d-ad0b-b2790512795e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.703845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded11809-e96a-475d-ad0b-b2790512795e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.708530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ba7cf-e33f-49c7-a761-51c20e41b9a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f89948b1e9480583555977ac15a6e0, user_id=54a1953b9908435592e6b38e7de9435c, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.711290 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ba7cf-e33f-49c7-a761-51c20e41b9a0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.726858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 791/1570] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:15 2022] PUT /identity/v3/projects/c1f89948b1e9480583555977ac15a6e0/users/54a1953b9908435592e6b38e7de9435c/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:15.735435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 780/1571] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:15.735435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded11809-e96a-475d-ad0b-b2790512795e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.738535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ba7cf-e33f-49c7-a761-51c20e41b9a0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.798318 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded11809-e96a-475d-ad0b-b2790512795e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.803345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ba7cf-e33f-49c7-a761-51c20e41b9a0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.803562 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9622113-5ceb-48a4-b596-e4b9ff9ff251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.804215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9622113-5ceb-48a4-b596-e4b9ff9ff251 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.804564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9622113-5ceb-48a4-b596-e4b9ff9ff251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.804736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9622113-5ceb-48a4-b596-e4b9ff9ff251 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.805508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9622113-5ceb-48a4-b596-e4b9ff9ff251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.807465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9622113-5ceb-48a4-b596-e4b9ff9ff251 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.808377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c3a77-987f-4310-bd99-431d39ff4488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.809064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c3a77-987f-4310-bd99-431d39ff4488 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.809361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c3a77-987f-4310-bd99-431d39ff4488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.809637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c3a77-987f-4310-bd99-431d39ff4488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.810407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c3a77-987f-4310-bd99-431d39ff4488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.812794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c3a77-987f-4310-bd99-431d39ff4488 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.824904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 792/1572] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:15.825776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c3a77-987f-4310-bd99-431d39ff4488 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.836708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c3a77-987f-4310-bd99-431d39ff4488 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.839750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 781/1573] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:15.840760 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9622113-5ceb-48a4-b596-e4b9ff9ff251 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.845086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea3fe8-9098-490a-b0a0-4fb23d71ec93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.845754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea3fe8-9098-490a-b0a0-4fb23d71ec93 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.846332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea3fe8-9098-490a-b0a0-4fb23d71ec93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.846465 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea3fe8-9098-490a-b0a0-4fb23d71ec93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.847307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea3fe8-9098-490a-b0a0-4fb23d71ec93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.849314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea3fe8-9098-490a-b0a0-4fb23d71ec93 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.889473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 793/1574] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:15.890487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea3fe8-9098-490a-b0a0-4fb23d71ec93 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:15.915529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9622113-5ceb-48a4-b596-e4b9ff9ff251 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.943737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:15.957011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea3fe8-9098-490a-b0a0-4fb23d71ec93 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:15.961663 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb3eae-8761-4903-8b44-def9e104c6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.962322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3eae-8761-4903-8b44-def9e104c6a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.962634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3eae-8761-4903-8b44-def9e104c6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.963266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3eae-8761-4903-8b44-def9e104c6a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.963957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb3eae-8761-4903-8b44-def9e104c6a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.965899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb3eae-8761-4903-8b44-def9e104c6a4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.980363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:15.980844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a14215-0b53-4fd7-88d4-f0dd7a6f95ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:15.981549 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a14215-0b53-4fd7-88d4-f0dd7a6f95ba admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:15.981873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a14215-0b53-4fd7-88d4-f0dd7a6f95ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:15.982258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a14215-0b53-4fd7-88d4-f0dd7a6f95ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:15.982984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a14215-0b53-4fd7-88d4-f0dd7a6f95ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:15.983319 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a14215-0b53-4fd7-88d4-f0dd7a6f95ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:15.985225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a14215-0b53-4fd7-88d4-f0dd7a6f95ba admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:15.994682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 794/1575] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:15.995294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb3eae-8761-4903-8b44-def9e104c6a4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.001569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 782/1576] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:16.007526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a14215-0b53-4fd7-88d4-f0dd7a6f95ba admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.019064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a14215-0b53-4fd7-88d4-f0dd7a6f95ba admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.069020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb3eae-8761-4903-8b44-def9e104c6a4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.073976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22afd8f-dd10-46aa-bff7-4735f320f780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.074685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22afd8f-dd10-46aa-bff7-4735f320f780 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.074973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22afd8f-dd10-46aa-bff7-4735f320f780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.075249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22afd8f-dd10-46aa-bff7-4735f320f780 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.075957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22afd8f-dd10-46aa-bff7-4735f320f780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.077889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22afd8f-dd10-46aa-bff7-4735f320f780 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.079226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88066aa-005b-4a39-ad4f-7598b15548e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.079932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88066aa-005b-4a39-ad4f-7598b15548e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.080220 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88066aa-005b-4a39-ad4f-7598b15548e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.080625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88066aa-005b-4a39-ad4f-7598b15548e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.081419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88066aa-005b-4a39-ad4f-7598b15548e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.083429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88066aa-005b-4a39-ad4f-7598b15548e2 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.111535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 783/1577] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:16 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:16.111758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 795/1578] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:15 2022] POST /identity/v3/projects => generated 405 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:16.111758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22afd8f-dd10-46aa-bff7-4735f320f780 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.111937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1cf6b-dc23-4556-8fa3-305ed888d600 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.111937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1cf6b-dc23-4556-8fa3-305ed888d600 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.111937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1cf6b-dc23-4556-8fa3-305ed888d600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.138750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-09f1cf6b-dc23-4556-8fa3-305ed888d600 None None] MFA Rules not processed for user `54a1953b9908435592e6b38e7de9435c`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:16.139999 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-09f1cf6b-dc23-4556-8fa3-305ed888d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:16.175485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22afd8f-dd10-46aa-bff7-4735f320f780 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.180842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec1e16-9165-456d-8e16-adc055ce2221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.181739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec1e16-9165-456d-8e16-adc055ce2221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.182019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec1e16-9165-456d-8e16-adc055ce2221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.182292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec1e16-9165-456d-8e16-adc055ce2221 admin admin] PATH_INFO: `/v3/projects/702000a5942b41689821164e169520d8/users/229771b215b3492abc2c5ec38ff0d929/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.198068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec1e16-9165-456d-8e16-adc055ce2221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702000a5942b41689821164e169520d8, user_id=229771b215b3492abc2c5ec38ff0d929, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.200805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec1e16-9165-456d-8e16-adc055ce2221 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.215786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 784/1579] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:16.216626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1cf6b-dc23-4556-8fa3-305ed888d600 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.217264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 796/1580] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:16 2022] PUT /identity/v3/projects/702000a5942b41689821164e169520d8/users/229771b215b3492abc2c5ec38ff0d929/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:16.217949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec1e16-9165-456d-8e16-adc055ce2221 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.228278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1cf6b-dc23-4556-8fa3-305ed888d600 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.269869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:16.284626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec1e16-9165-456d-8e16-adc055ce2221 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.289377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4b5b6-e361-4767-b71b-13a3cb48803a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.289587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6b522-ebd0-4261-9dd4-c2153a16e292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.290067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4b5b6-e361-4767-b71b-13a3cb48803a admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.290292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6b522-ebd0-4261-9dd4-c2153a16e292 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.290406 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4b5b6-e361-4767-b71b-13a3cb48803a admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.290579 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6b522-ebd0-4261-9dd4-c2153a16e292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.290734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4b5b6-e361-4767-b71b-13a3cb48803a admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.290901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6b522-ebd0-4261-9dd4-c2153a16e292 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.291318 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4b5b6-e361-4767-b71b-13a3cb48803a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.291553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6b522-ebd0-4261-9dd4-c2153a16e292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.293346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4b5b6-e361-4767-b71b-13a3cb48803a admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.293566 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6b522-ebd0-4261-9dd4-c2153a16e292 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.303077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:16.315830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 785/1581] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:16 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:16.316560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4b5b6-e361-4767-b71b-13a3cb48803a admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.320574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 797/1582] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:16 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:16.320574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6b522-ebd0-4261-9dd4-c2153a16e292 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.328118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4b5b6-e361-4767-b71b-13a3cb48803a admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.330905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6b522-ebd0-4261-9dd4-c2153a16e292 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.333306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f64a12-1e04-4d18-ad12-0d8069fc7085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.333984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f64a12-1e04-4d18-ad12-0d8069fc7085 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.334262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f64a12-1e04-4d18-ad12-0d8069fc7085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.334534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f64a12-1e04-4d18-ad12-0d8069fc7085 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.335251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f64a12-1e04-4d18-ad12-0d8069fc7085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.336548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa2c90-ad8e-464c-aa81-d4e4466dd6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.337247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f64a12-1e04-4d18-ad12-0d8069fc7085 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.337355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2c90-ad8e-464c-aa81-d4e4466dd6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.337597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2c90-ad8e-464c-aa81-d4e4466dd6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.338231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2c90-ad8e-464c-aa81-d4e4466dd6e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.338872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa2c90-ad8e-464c-aa81-d4e4466dd6e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.340828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa2c90-ad8e-464c-aa81-d4e4466dd6e6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.357719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 786/1583] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:16 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:16.358396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f64a12-1e04-4d18-ad12-0d8069fc7085 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.368616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f64a12-1e04-4d18-ad12-0d8069fc7085 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.373230 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887706-4969-47e8-9c63-09708b846d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.373823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 798/1584] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:16 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:16.373936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887706-4969-47e8-9c63-09708b846d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.374168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887706-4969-47e8-9c63-09708b846d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.374442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887706-4969-47e8-9c63-09708b846d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.374753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa2c90-ad8e-464c-aa81-d4e4466dd6e6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.375197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e887706-4969-47e8-9c63-09708b846d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.376796 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e887706-4969-47e8-9c63-09708b846d9c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.391914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 787/1585] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.391914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e4713-7720-4924-a486-b86d188e7d9e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.391914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e4713-7720-4924-a486-b86d188e7d9e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.391914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e4713-7720-4924-a486-b86d188e7d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.408935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-233e4713-7720-4924-a486-b86d188e7d9e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:16.410411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-233e4713-7720-4924-a486-b86d188e7d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:16.444077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa2c90-ad8e-464c-aa81-d4e4466dd6e6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.449341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555984d3-2ebf-486a-9aac-3287fe2e9d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.450040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555984d3-2ebf-486a-9aac-3287fe2e9d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.450324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555984d3-2ebf-486a-9aac-3287fe2e9d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.450623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555984d3-2ebf-486a-9aac-3287fe2e9d04 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.451361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555984d3-2ebf-486a-9aac-3287fe2e9d04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.453396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555984d3-2ebf-486a-9aac-3287fe2e9d04 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.475437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 799/1586] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:16 2022] POST /identity/v3/users => generated 303 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:16.475894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 788/1587] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:16.476681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555984d3-2ebf-486a-9aac-3287fe2e9d04 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.476789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e4713-7720-4924-a486-b86d188e7d9e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.488168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e4713-7720-4924-a486-b86d188e7d9e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.490584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555984d3-2ebf-486a-9aac-3287fe2e9d04 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.492780 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05421c4-5498-428a-b25b-289af58e23fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.493473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05421c4-5498-428a-b25b-289af58e23fb admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.493758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05421c4-5498-428a-b25b-289af58e23fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.494038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05421c4-5498-428a-b25b-289af58e23fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.494884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05421c4-5498-428a-b25b-289af58e23fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.496080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd5048-ee33-449a-8955-c6e084bbcde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.496491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05421c4-5498-428a-b25b-289af58e23fb admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.496747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd5048-ee33-449a-8955-c6e084bbcde1 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.497023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd5048-ee33-449a-8955-c6e084bbcde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.497291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd5048-ee33-449a-8955-c6e084bbcde1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.497983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd5048-ee33-449a-8955-c6e084bbcde1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.499917 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd5048-ee33-449a-8955-c6e084bbcde1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.507103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 789/1588] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.507953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05421c4-5498-428a-b25b-289af58e23fb admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.519330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05421c4-5498-428a-b25b-289af58e23fb admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.524033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c193e3-852b-4ef2-8bd0-74f19544b9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.524715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c193e3-852b-4ef2-8bd0-74f19544b9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.525003 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c193e3-852b-4ef2-8bd0-74f19544b9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.525277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c193e3-852b-4ef2-8bd0-74f19544b9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.526007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c193e3-852b-4ef2-8bd0-74f19544b9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.527689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c193e3-852b-4ef2-8bd0-74f19544b9ee admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.528877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 800/1589] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:16 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:16.531530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd5048-ee33-449a-8955-c6e084bbcde1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.542362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 790/1590] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.543301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c193e3-852b-4ef2-8bd0-74f19544b9ee admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.553399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c193e3-852b-4ef2-8bd0-74f19544b9ee admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.597670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd5048-ee33-449a-8955-c6e084bbcde1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.603350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ce16d-24d5-4962-8100-3bcf8b3c7469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.604034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ce16d-24d5-4962-8100-3bcf8b3c7469 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.604314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ce16d-24d5-4962-8100-3bcf8b3c7469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.604584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ce16d-24d5-4962-8100-3bcf8b3c7469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.605314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ce16d-24d5-4962-8100-3bcf8b3c7469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.606995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ce16d-24d5-4962-8100-3bcf8b3c7469 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.611768 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3ddc1-819b-4f77-8b5a-36f8736e8112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.612481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3ddc1-819b-4f77-8b5a-36f8736e8112 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.612766 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3ddc1-819b-4f77-8b5a-36f8736e8112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.613036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3ddc1-819b-4f77-8b5a-36f8736e8112 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.613760 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3ddc1-819b-4f77-8b5a-36f8736e8112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.615751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3ddc1-819b-4f77-8b5a-36f8736e8112 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.617774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 801/1591] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.618644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ce16d-24d5-4962-8100-3bcf8b3c7469 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.629015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ce16d-24d5-4962-8100-3bcf8b3c7469 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.634109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360e4a2-facf-4917-98a8-eb3c319d362e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.634819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360e4a2-facf-4917-98a8-eb3c319d362e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.635101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360e4a2-facf-4917-98a8-eb3c319d362e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.635377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360e4a2-facf-4917-98a8-eb3c319d362e admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.636104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360e4a2-facf-4917-98a8-eb3c319d362e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.639931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 791/1592] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:16 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:16.639931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3ddc1-819b-4f77-8b5a-36f8736e8112 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.640174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360e4a2-facf-4917-98a8-eb3c319d362e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.648933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 802/1593] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.649747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360e4a2-facf-4917-98a8-eb3c319d362e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.650333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3ddc1-819b-4f77-8b5a-36f8736e8112 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.656503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c608647-0682-4146-a27d-dc3501f1abb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.657178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c608647-0682-4146-a27d-dc3501f1abb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.657457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c608647-0682-4146-a27d-dc3501f1abb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.657730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c608647-0682-4146-a27d-dc3501f1abb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.658500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c608647-0682-4146-a27d-dc3501f1abb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.660463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c608647-0682-4146-a27d-dc3501f1abb1 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.666023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:16.672391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 792/1594] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.677730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c608647-0682-4146-a27d-dc3501f1abb1 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.680040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:16.690035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c608647-0682-4146-a27d-dc3501f1abb1 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.695192 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf276d-532f-4028-b876-7f5ac481c815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.695879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf276d-532f-4028-b876-7f5ac481c815 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.696170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf276d-532f-4028-b876-7f5ac481c815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.696492 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf276d-532f-4028-b876-7f5ac481c815 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.697248 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf276d-532f-4028-b876-7f5ac481c815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.698899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf276d-532f-4028-b876-7f5ac481c815 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.710008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 793/1595] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.710821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360e4a2-facf-4917-98a8-eb3c319d362e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.710979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf276d-532f-4028-b876-7f5ac481c815 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.715133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3fb39-73ad-4a91-980a-6af760adfe7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.715790 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3fb39-73ad-4a91-980a-6af760adfe7d service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.716075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3fb39-73ad-4a91-980a-6af760adfe7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.716347 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3fb39-73ad-4a91-980a-6af760adfe7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.721066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf276d-532f-4028-b876-7f5ac481c815 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.725768 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ec49f-6ef0-42d7-98b0-dba594bbef34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.726509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ec49f-6ef0-42d7-98b0-dba594bbef34 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.726671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3fb39-73ad-4a91-980a-6af760adfe7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.726830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ec49f-6ef0-42d7-98b0-dba594bbef34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.727103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ec49f-6ef0-42d7-98b0-dba594bbef34 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.727818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ec49f-6ef0-42d7-98b0-dba594bbef34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.728082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3fb39-73ad-4a91-980a-6af760adfe7d service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.729754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ec49f-6ef0-42d7-98b0-dba594bbef34 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.738779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 803/1596] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:16.739716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3fb39-73ad-4a91-980a-6af760adfe7d service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.750033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3fb39-73ad-4a91-980a-6af760adfe7d service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.751004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 794/1597] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:16 2022] POST /identity/v3/users => generated 295 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:16.754965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ec49f-6ef0-42d7-98b0-dba594bbef34 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.755870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d25eb-92fa-400f-a03c-32e6486dd1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.756619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d25eb-92fa-400f-a03c-32e6486dd1ad admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.757392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d25eb-92fa-400f-a03c-32e6486dd1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.757392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d25eb-92fa-400f-a03c-32e6486dd1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.758369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d25eb-92fa-400f-a03c-32e6486dd1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.759639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d25eb-92fa-400f-a03c-32e6486dd1ad admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.765526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ec49f-6ef0-42d7-98b0-dba594bbef34 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.771427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ae631-160e-406f-bdce-bed8b486b2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.772135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ae631-160e-406f-bdce-bed8b486b2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.772319 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 804/1598] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.772441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ae631-160e-406f-bdce-bed8b486b2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.772685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ae631-160e-406f-bdce-bed8b486b2f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.773339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d25eb-92fa-400f-a03c-32e6486dd1ad admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.773523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ae631-160e-406f-bdce-bed8b486b2f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.775274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ae631-160e-406f-bdce-bed8b486b2f4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.786887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d25eb-92fa-400f-a03c-32e6486dd1ad admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.789379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 795/1599] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.790490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ae631-160e-406f-bdce-bed8b486b2f4 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.796340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3bc6-3ff6-406a-b4a9-6793a171440b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.797037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3bc6-3ff6-406a-b4a9-6793a171440b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.797322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3bc6-3ff6-406a-b4a9-6793a171440b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.797599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3bc6-3ff6-406a-b4a9-6793a171440b admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.798505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a3bc6-3ff6-406a-b4a9-6793a171440b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.800189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a3bc6-3ff6-406a-b4a9-6793a171440b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.802505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ae631-160e-406f-bdce-bed8b486b2f4 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.809756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17b75f-eff3-4703-92ad-aecf05330eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.810966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17b75f-eff3-4703-92ad-aecf05330eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.811428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17b75f-eff3-4703-92ad-aecf05330eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.811859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17b75f-eff3-4703-92ad-aecf05330eb3 admin admin] PATH_INFO: `/v3/projects/f57102c066054370a153fb18dc4bc428/users/7900ba750c5f4e9faa69baaed1fd95d7/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.813830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 805/1600] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.815155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3bc6-3ff6-406a-b4a9-6793a171440b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.828907 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3bc6-3ff6-406a-b4a9-6793a171440b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.832054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17b75f-eff3-4703-92ad-aecf05330eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f57102c066054370a153fb18dc4bc428, user_id=7900ba750c5f4e9faa69baaed1fd95d7, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.833106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: WARNING py.warnings [None req-cd17b75f-eff3-4703-92ad-aecf05330eb3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:58:16.833106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: warnings.warn(msg) Jun 29 09:58:16.833106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: Jun 29 09:58:16.835146 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17b75f-eff3-4703-92ad-aecf05330eb3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.835663 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60852a7a-0700-4798-aae7-e90aa0d25c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.836548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60852a7a-0700-4798-aae7-e90aa0d25c13 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.836885 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60852a7a-0700-4798-aae7-e90aa0d25c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.837169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60852a7a-0700-4798-aae7-e90aa0d25c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.838592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60852a7a-0700-4798-aae7-e90aa0d25c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.840739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60852a7a-0700-4798-aae7-e90aa0d25c13 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.853692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 796/1601] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:16 2022] PUT /identity/v3/projects/f57102c066054370a153fb18dc4bc428/users/7900ba750c5f4e9faa69baaed1fd95d7/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:16.853929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 806/1602] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.854736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60852a7a-0700-4798-aae7-e90aa0d25c13 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.855478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17b75f-eff3-4703-92ad-aecf05330eb3 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.870342 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60852a7a-0700-4798-aae7-e90aa0d25c13 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.935417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36e156-9c6b-46bd-b620-8300f4e45c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.936335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36e156-9c6b-46bd-b620-8300f4e45c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.936615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36e156-9c6b-46bd-b620-8300f4e45c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.936727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17b75f-eff3-4703-92ad-aecf05330eb3 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.936889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36e156-9c6b-46bd-b620-8300f4e45c4b admin admin] PATH_INFO: `/v3/projects/f5ea4e19435c41809a46335000653767/users/d081e7eeef1b4aa49f6598b9b5307c03/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.945168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f784a-e949-4097-82d5-7ee72f62e611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:16.945366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f784a-e949-4097-82d5-7ee72f62e611 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:16.945703 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f784a-e949-4097-82d5-7ee72f62e611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:16.946041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f784a-e949-4097-82d5-7ee72f62e611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:16.946883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f784a-e949-4097-82d5-7ee72f62e611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.948781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f784a-e949-4097-82d5-7ee72f62e611 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.958915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36e156-9c6b-46bd-b620-8300f4e45c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ea4e19435c41809a46335000653767, user_id=d081e7eeef1b4aa49f6598b9b5307c03, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:16.965322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 797/1603] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:16.965322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f784a-e949-4097-82d5-7ee72f62e611 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:16.965842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36e156-9c6b-46bd-b620-8300f4e45c4b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:16.976388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f784a-e949-4097-82d5-7ee72f62e611 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:16.984172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 807/1604] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:16 2022] PUT /identity/v3/projects/f5ea4e19435c41809a46335000653767/users/d081e7eeef1b4aa49f6598b9b5307c03/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:16.991990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36e156-9c6b-46bd-b620-8300f4e45c4b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:17.055200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1429a0-5e0d-471e-bc59-bab2710bfa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:17.056530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1429a0-5e0d-471e-bc59-bab2710bfa1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.056915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1429a0-5e0d-471e-bc59-bab2710bfa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.057440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1429a0-5e0d-471e-bc59-bab2710bfa1e admin admin] PATH_INFO: `/v3/projects/d195d5e7f60544b48e25f0bedbbe8731/users/aba55f5cda1441249e414223ba8f0269/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.071879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36e156-9c6b-46bd-b620-8300f4e45c4b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:17.079254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669fe99-a595-4c27-b092-2e1416866697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:17.080191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1429a0-5e0d-471e-bc59-bab2710bfa1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d195d5e7f60544b48e25f0bedbbe8731, user_id=aba55f5cda1441249e414223ba8f0269, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:17.080622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669fe99-a595-4c27-b092-2e1416866697 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.081245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669fe99-a595-4c27-b092-2e1416866697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.081690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669fe99-a595-4c27-b092-2e1416866697 admin admin] PATH_INFO: `/v3/projects/e10f49f1727f4327be986e4f7e131200/users/57605be4d856407f9ac4828ee1fef041/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.083688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1429a0-5e0d-471e-bc59-bab2710bfa1e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:17.100362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669fe99-a595-4c27-b092-2e1416866697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e10f49f1727f4327be986e4f7e131200, user_id=57605be4d856407f9ac4828ee1fef041, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:17.102943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 798/1605] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:16 2022] PUT /identity/v3/projects/d195d5e7f60544b48e25f0bedbbe8731/users/aba55f5cda1441249e414223ba8f0269/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:17.103443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669fe99-a595-4c27-b092-2e1416866697 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:17.104732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1429a0-5e0d-471e-bc59-bab2710bfa1e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:17.115722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:17.127748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 808/1606] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:16 2022] PUT /identity/v3/projects/e10f49f1727f4327be986e4f7e131200/users/57605be4d856407f9ac4828ee1fef041/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:17.129734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d96a7a-03e7-4a2f-8a6b-e5567700e767 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.130183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d96a7a-03e7-4a2f-8a6b-e5567700e767 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.130543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d96a7a-03e7-4a2f-8a6b-e5567700e767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.131442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:17.161477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b8d96a7a-03e7-4a2f-8a6b-e5567700e767 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:17.163870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b8d96a7a-03e7-4a2f-8a6b-e5567700e767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:17.204700 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1429a0-5e0d-471e-bc59-bab2710bfa1e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:17.266477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 809/1607] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:17.268251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d96a7a-03e7-4a2f-8a6b-e5567700e767 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:17.279308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d96a7a-03e7-4a2f-8a6b-e5567700e767 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:17.285453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643edcab-6e8b-4a62-a8f6-711e0b1ca796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:17.286474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643edcab-6e8b-4a62-a8f6-711e0b1ca796 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.286854 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643edcab-6e8b-4a62-a8f6-711e0b1ca796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.287339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643edcab-6e8b-4a62-a8f6-711e0b1ca796 admin admin] PATH_INFO: `/v3/projects/61d51cf573ee400ebbb92c3d68fcbefe/users/0081f49018be43a387976158005f0552/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.304534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b9199-14fe-43c2-a806-24948e13a0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:17.305456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b9199-14fe-43c2-a806-24948e13a0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.305807 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b9199-14fe-43c2-a806-24948e13a0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.306156 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b9199-14fe-43c2-a806-24948e13a0e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.306948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b9199-14fe-43c2-a806-24948e13a0e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:17.308681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b9199-14fe-43c2-a806-24948e13a0e5 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:17.309867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643edcab-6e8b-4a62-a8f6-711e0b1ca796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d51cf573ee400ebbb92c3d68fcbefe, user_id=0081f49018be43a387976158005f0552, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:17.317680 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643edcab-6e8b-4a62-a8f6-711e0b1ca796 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:17.322536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 799/1608] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:17.323425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b9199-14fe-43c2-a806-24948e13a0e5 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:17.335395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b9199-14fe-43c2-a806-24948e13a0e5 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:17.337458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 810/1609] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:17 2022] PUT /identity/v3/projects/61d51cf573ee400ebbb92c3d68fcbefe/users/0081f49018be43a387976158005f0552/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:17.349387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f6fa9-77e6-4183-95dc-a4410622a410 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.349387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f6fa9-77e6-4183-95dc-a4410622a410 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.349387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f6fa9-77e6-4183-95dc-a4410622a410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.380326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-891f6fa9-77e6-4183-95dc-a4410622a410 None None] MFA Rules not processed for user `d081e7eeef1b4aa49f6598b9b5307c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:17.381489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-891f6fa9-77e6-4183-95dc-a4410622a410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:17.407450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e80098-c62f-4ef9-b519-f2d8a25ba70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:17.408803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e80098-c62f-4ef9-b519-f2d8a25ba70e admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.409426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e80098-c62f-4ef9-b519-f2d8a25ba70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.409852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e80098-c62f-4ef9-b519-f2d8a25ba70e admin admin] PATH_INFO: `/v3/projects/c1ce60b7a32544f2b60da5efbe0d1e85/users/cc4fd9d5664547e6950f468489257f50/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.434135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e80098-c62f-4ef9-b519-f2d8a25ba70e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ce60b7a32544f2b60da5efbe0d1e85, user_id=cc4fd9d5664547e6950f468489257f50, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:17.437516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e80098-c62f-4ef9-b519-f2d8a25ba70e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:17.456057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 800/1610] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:17 2022] PUT /identity/v3/projects/c1ce60b7a32544f2b60da5efbe0d1e85/users/cc4fd9d5664547e6950f468489257f50/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:17.457452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79fbb3-f971-4ece-88ca-d20d0aa0717c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.457822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79fbb3-f971-4ece-88ca-d20d0aa0717c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.457991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79fbb3-f971-4ece-88ca-d20d0aa0717c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.463155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 811/1611] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:17.465355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126530a-b83c-4cc3-856a-790bc8230a04 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.465693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126530a-b83c-4cc3-856a-790bc8230a04 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.465971 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126530a-b83c-4cc3-856a-790bc8230a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.501357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e126530a-b83c-4cc3-856a-790bc8230a04 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:17.503005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e126530a-b83c-4cc3-856a-790bc8230a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:17.508587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fd79fbb3-f971-4ece-88ca-d20d0aa0717c None None] MFA Rules not processed for user `aba55f5cda1441249e414223ba8f0269`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:17.510135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fd79fbb3-f971-4ece-88ca-d20d0aa0717c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:17.547918 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:17.583872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 812/1612] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:17.585191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126530a-b83c-4cc3-856a-790bc8230a04 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:17.596543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:17.608750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 801/1613] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:17.608750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79fbb3-f971-4ece-88ca-d20d0aa0717c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:17.621441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79fbb3-f971-4ece-88ca-d20d0aa0717c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:17.632005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec507f08-23c7-42a7-802f-f0c9001be0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:17.633229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec507f08-23c7-42a7-802f-f0c9001be0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.633669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec507f08-23c7-42a7-802f-f0c9001be0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.633948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec507f08-23c7-42a7-802f-f0c9001be0f5 admin admin] PATH_INFO: `/v3/projects/c695a228acc84e93baf309bec5e489c4/users/63ba0760d13e4bbda0650de803ef46ea/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.667969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec507f08-23c7-42a7-802f-f0c9001be0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c695a228acc84e93baf309bec5e489c4, user_id=63ba0760d13e4bbda0650de803ef46ea, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:17.671062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec507f08-23c7-42a7-802f-f0c9001be0f5 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:17.679559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126530a-b83c-4cc3-856a-790bc8230a04 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:17.683840 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69469e96-3766-4bc1-86b9-b9a0f93593ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:17.684814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469e96-3766-4bc1-86b9-b9a0f93593ed service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.685138 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469e96-3766-4bc1-86b9-b9a0f93593ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.685414 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469e96-3766-4bc1-86b9-b9a0f93593ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.691368 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 802/1614] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:17 2022] PUT /identity/v3/projects/c695a228acc84e93baf309bec5e489c4/users/63ba0760d13e4bbda0650de803ef46ea/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:17.693287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76f098-9b93-4c4c-a011-78cac129bed6 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.693744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76f098-9b93-4c4c-a011-78cac129bed6 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.694800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76f098-9b93-4c4c-a011-78cac129bed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.698134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69469e96-3766-4bc1-86b9-b9a0f93593ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:17.699763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69469e96-3766-4bc1-86b9-b9a0f93593ed service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:17.711490 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 813/1615] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:17.712160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69469e96-3766-4bc1-86b9-b9a0f93593ed service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:17.724692 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69469e96-3766-4bc1-86b9-b9a0f93593ed service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:17.734909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-1f76f098-9b93-4c4c-a011-78cac129bed6 None None] MFA Rules not processed for user `0081f49018be43a387976158005f0552`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:17.736776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-1f76f098-9b93-4c4c-a011-78cac129bed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:17.799238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb508d69-c5b4-4293-9887-99be639ab5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:17.800225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb508d69-c5b4-4293-9887-99be639ab5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.800687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb508d69-c5b4-4293-9887-99be639ab5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.801199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb508d69-c5b4-4293-9887-99be639ab5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.802619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb508d69-c5b4-4293-9887-99be639ab5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:17.804915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb508d69-c5b4-4293-9887-99be639ab5eb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:17.816751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 803/1616] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:17 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:17.820575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76f098-9b93-4c4c-a011-78cac129bed6 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:17.821279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 814/1617] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:17.822736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb508d69-c5b4-4293-9887-99be639ab5eb admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:17.897473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76f098-9b93-4c4c-a011-78cac129bed6 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:17.901883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d248f-2ddc-4fc2-88d9-555b5766c9de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:17.902242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d248f-2ddc-4fc2-88d9-555b5766c9de service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.902595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d248f-2ddc-4fc2-88d9-555b5766c9de service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.902874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d248f-2ddc-4fc2-88d9-555b5766c9de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.916716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb508d69-c5b4-4293-9887-99be639ab5eb admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:17.922469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af21e3-359d-4a1a-be61-e65e70ea28fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:17.923461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af21e3-359d-4a1a-be61-e65e70ea28fe service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.924352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af21e3-359d-4a1a-be61-e65e70ea28fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.924936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af21e3-359d-4a1a-be61-e65e70ea28fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.937186 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af21e3-359d-4a1a-be61-e65e70ea28fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:17.939321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af21e3-359d-4a1a-be61-e65e70ea28fe service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:17.953249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 815/1618] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:17.955318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ca483-8ac5-449c-a847-2f3875327fdd None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.956051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ca483-8ac5-449c-a847-2f3875327fdd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.956536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ca483-8ac5-449c-a847-2f3875327fdd None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.958885 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 816/1619] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jun 29 09:58:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:58:17.960253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:17.961657 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dcbea-4a96-46a2-8268-bbc2a5ad58a8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:17.962221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dcbea-4a96-46a2-8268-bbc2a5ad58a8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:17.963130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dcbea-4a96-46a2-8268-bbc2a5ad58a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:17.971459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d248f-2ddc-4fc2-88d9-555b5766c9de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:17.973502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d248f-2ddc-4fc2-88d9-555b5766c9de service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:17.985635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 804/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:17 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:17.986733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d248f-2ddc-4fc2-88d9-555b5766c9de service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:17.998386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d248f-2ddc-4fc2-88d9-555b5766c9de service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:18.000403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-205dcbea-4a96-46a2-8268-bbc2a5ad58a8 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:18.002057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-205dcbea-4a96-46a2-8268-bbc2a5ad58a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:18.012204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73c1f2-be01-4223-ac6f-59b30828e4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:18.013273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73c1f2-be01-4223-ac6f-59b30828e4d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:18.013560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73c1f2-be01-4223-ac6f-59b30828e4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:18.017129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73c1f2-be01-4223-ac6f-59b30828e4d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:18.017286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 817/1621] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:18.017398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73c1f2-be01-4223-ac6f-59b30828e4d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:18.017398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73c1f2-be01-4223-ac6f-59b30828e4d5 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:18.017568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dcbea-4a96-46a2-8268-bbc2a5ad58a8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:18.041293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dcbea-4a96-46a2-8268-bbc2a5ad58a8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:18.052369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2117307-08a0-49a4-bb47-630c914f7d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:18.053351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117307-08a0-49a4-bb47-630c914f7d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:18.053625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117307-08a0-49a4-bb47-630c914f7d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:18.053901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117307-08a0-49a4-bb47-630c914f7d19 admin admin] PATH_INFO: `/v3/projects/c1ce60b7a32544f2b60da5efbe0d1e85/users/cc4fd9d5664547e6950f468489257f50/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:18.061265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2117307-08a0-49a4-bb47-630c914f7d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ce60b7a32544f2b60da5efbe0d1e85, user_id=cc4fd9d5664547e6950f468489257f50, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:18.065508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2117307-08a0-49a4-bb47-630c914f7d19 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:18.075767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 805/1622] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:17 2022] POST /identity/v3/projects => generated 403 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:18.077554 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c9c56-fe2c-4041-ae7e-03a9b2b12c7e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:18.078107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c9c56-fe2c-4041-ae7e-03a9b2b12c7e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:18.078684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c9c56-fe2c-4041-ae7e-03a9b2b12c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:18.104019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 818/1623] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:18 2022] PUT /identity/v3/projects/c1ce60b7a32544f2b60da5efbe0d1e85/users/cc4fd9d5664547e6950f468489257f50/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:18.104282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-697c9c56-fe2c-4041-ae7e-03a9b2b12c7e None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:18.105053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2117307-08a0-49a4-bb47-630c914f7d19 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:18.106518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-697c9c56-fe2c-4041-ae7e-03a9b2b12c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:18.139379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:18.167866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2117307-08a0-49a4-bb47-630c914f7d19 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:18.173830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c3297-6160-4410-b32b-699938e93329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:18.174687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c3297-6160-4410-b32b-699938e93329 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:18.176374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c3297-6160-4410-b32b-699938e93329 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:18.176666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c3297-6160-4410-b32b-699938e93329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:18.275565 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 806/1624] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jun 29 09:58:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:18.276170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c9c56-fe2c-4041-ae7e-03a9b2b12c7e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:18.320119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c3297-6160-4410-b32b-699938e93329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:18.324346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c3297-6160-4410-b32b-699938e93329 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:18.357190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 819/1625] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:18.358378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3af598-55bf-47c4-9f6c-33ccdf1e5e92 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:18.358710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3af598-55bf-47c4-9f6c-33ccdf1e5e92 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:18.358978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3af598-55bf-47c4-9f6c-33ccdf1e5e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:18.389421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c9c56-fe2c-4041-ae7e-03a9b2b12c7e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:18.404814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803da4df-1b7f-4699-b6cc-1099217cd84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:18.405531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803da4df-1b7f-4699-b6cc-1099217cd84c admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:18.405907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803da4df-1b7f-4699-b6cc-1099217cd84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:18.406921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803da4df-1b7f-4699-b6cc-1099217cd84c admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:18.407055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803da4df-1b7f-4699-b6cc-1099217cd84c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:18.411721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803da4df-1b7f-4699-b6cc-1099217cd84c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:18.419129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4b3af598-55bf-47c4-9f6c-33ccdf1e5e92 None None] MFA Rules not processed for user `cc4fd9d5664547e6950f468489257f50`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:18.420333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4b3af598-55bf-47c4-9f6c-33ccdf1e5e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:18.465794 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 807/1626] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:18 2022] POST /identity/v3/users => generated 308 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:18.467165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803da4df-1b7f-4699-b6cc-1099217cd84c admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:18.487563 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803da4df-1b7f-4699-b6cc-1099217cd84c admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:18.497285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f7fba-b6a3-43ec-a2e8-79161c5d2592 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:18.498354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f7fba-b6a3-43ec-a2e8-79161c5d2592 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:18.498816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f7fba-b6a3-43ec-a2e8-79161c5d2592 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:18.499259 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f7fba-b6a3-43ec-a2e8-79161c5d2592 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:18.584562 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:18.607340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 820/1627] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:18.609701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3af598-55bf-47c4-9f6c-33ccdf1e5e92 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:18.615464 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f7fba-b6a3-43ec-a2e8-79161c5d2592 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:18.618412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f7fba-b6a3-43ec-a2e8-79161c5d2592 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:18.634286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 808/1628] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:18.636095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f7fba-b6a3-43ec-a2e8-79161c5d2592 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:18.654717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3af598-55bf-47c4-9f6c-33ccdf1e5e92 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:18.698100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c7bf8-0d19-4374-aecf-7e0ec78ad132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:18.708392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c7bf8-0d19-4374-aecf-7e0ec78ad132 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:18.708777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c7bf8-0d19-4374-aecf-7e0ec78ad132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:18.709068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c7bf8-0d19-4374-aecf-7e0ec78ad132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:18.718034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c7bf8-0d19-4374-aecf-7e0ec78ad132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:18.721941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c7bf8-0d19-4374-aecf-7e0ec78ad132 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:18.752437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f7fba-b6a3-43ec-a2e8-79161c5d2592 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:18.758769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b4c68-d2e2-42ed-bc2b-653c17a1775e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:18.760296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b4c68-d2e2-42ed-bc2b-653c17a1775e service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:18.760834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b4c68-d2e2-42ed-bc2b-653c17a1775e service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:18.761631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b4c68-d2e2-42ed-bc2b-653c17a1775e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:18.766241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 821/1629] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:18.770797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c7bf8-0d19-4374-aecf-7e0ec78ad132 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:18.788577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c7bf8-0d19-4374-aecf-7e0ec78ad132 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:18.805722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e3945-19f9-4f66-a1dc-ca621a4bb94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:18.808103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e3945-19f9-4f66-a1dc-ca621a4bb94b admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:18.808623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e3945-19f9-4f66-a1dc-ca621a4bb94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:18.810946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e3945-19f9-4f66-a1dc-ca621a4bb94b admin admin] PATH_INFO: `/v3/projects/36c4a8a1e28c4d3e96da5ec0225a1b37/users/7aa74832ab7a4e9987e94287e990b362/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:18.857758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e3945-19f9-4f66-a1dc-ca621a4bb94b admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c4a8a1e28c4d3e96da5ec0225a1b37, user_id=7aa74832ab7a4e9987e94287e990b362, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:18.863942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e3945-19f9-4f66-a1dc-ca621a4bb94b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:18.900887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 822/1630] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:18 2022] PUT /identity/v3/projects/36c4a8a1e28c4d3e96da5ec0225a1b37/users/7aa74832ab7a4e9987e94287e990b362/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:58:18.902068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e3945-19f9-4f66-a1dc-ca621a4bb94b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:18.935476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b4c68-d2e2-42ed-bc2b-653c17a1775e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:18.937323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b4c68-d2e2-42ed-bc2b-653c17a1775e service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:18.965563 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 809/1631] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:18.975016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:19.034446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e3945-19f9-4f66-a1dc-ca621a4bb94b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:19.054702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54841512-52d4-4f74-8640-ceaf2fc99df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:19.055446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54841512-52d4-4f74-8640-ceaf2fc99df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:19.055794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54841512-52d4-4f74-8640-ceaf2fc99df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:19.056076 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54841512-52d4-4f74-8640-ceaf2fc99df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:19.056830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54841512-52d4-4f74-8640-ceaf2fc99df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:19.058784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54841512-52d4-4f74-8640-ceaf2fc99df3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:19.074873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 823/1632] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:58:19.079033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b4c68-d2e2-42ed-bc2b-653c17a1775e service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:19.099103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b4c68-d2e2-42ed-bc2b-653c17a1775e service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:19.108608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5661f14-0875-49d7-8988-6d2417ba88e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:19.109919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5661f14-0875-49d7-8988-6d2417ba88e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:19.111140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5661f14-0875-49d7-8988-6d2417ba88e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:19.111915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5661f14-0875-49d7-8988-6d2417ba88e4 admin admin] PATH_INFO: `/v3/projects/36c4a8a1e28c4d3e96da5ec0225a1b37/users/7aa74832ab7a4e9987e94287e990b362/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:19.115750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5661f14-0875-49d7-8988-6d2417ba88e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c4a8a1e28c4d3e96da5ec0225a1b37, user_id=7aa74832ab7a4e9987e94287e990b362, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:19.119598 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5661f14-0875-49d7-8988-6d2417ba88e4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:19.138406 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 810/1633] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:19 2022] PUT /identity/v3/projects/36c4a8a1e28c4d3e96da5ec0225a1b37/users/7aa74832ab7a4e9987e94287e990b362/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:19.150327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f2cfa-dc92-461b-96e7-f6917e753b90 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:19.150749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f2cfa-dc92-461b-96e7-f6917e753b90 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:19.151242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f2cfa-dc92-461b-96e7-f6917e753b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:19.218178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-608f2cfa-dc92-461b-96e7-f6917e753b90 None None] MFA Rules not processed for user `7aa74832ab7a4e9987e94287e990b362`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:19.219468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-608f2cfa-dc92-461b-96e7-f6917e753b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:19.414647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 824/1634] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:19 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:19.424051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5661f14-0875-49d7-8988-6d2417ba88e4 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:19.543445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5661f14-0875-49d7-8988-6d2417ba88e4 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:19.548319 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fc60c-1d1c-4784-850a-3ae75c2dfdd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:19.552311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fc60c-1d1c-4784-850a-3ae75c2dfdd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:19.552707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fc60c-1d1c-4784-850a-3ae75c2dfdd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:19.553105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fc60c-1d1c-4784-850a-3ae75c2dfdd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:19.572616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fc60c-1d1c-4784-850a-3ae75c2dfdd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:19.574285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fc60c-1d1c-4784-850a-3ae75c2dfdd8 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:19.592848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 811/1635] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:19 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:19.725604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d14d7-9b25-4830-b247-41f8728de7d3 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:19.725902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d14d7-9b25-4830-b247-41f8728de7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:19.726267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d14d7-9b25-4830-b247-41f8728de7d3 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:19.728278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 825/1636] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jun 29 09:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:58:19.735304 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b82c12-e569-4bef-8148-4c024de2246b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:19.735605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b82c12-e569-4bef-8148-4c024de2246b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:19.735884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b82c12-e569-4bef-8148-4c024de2246b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:19.761227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-62b82c12-e569-4bef-8148-4c024de2246b None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:19.762178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-62b82c12-e569-4bef-8148-4c024de2246b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:19.896117 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:19.896935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d14d7-9b25-4830-b247-41f8728de7d3 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:20.036729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d14d7-9b25-4830-b247-41f8728de7d3 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:20.050213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 812/1637] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 09:58:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:20.051561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069aaee-502d-46f9-97af-2cabf3b039d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:20.054157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069aaee-502d-46f9-97af-2cabf3b039d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:20.054157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069aaee-502d-46f9-97af-2cabf3b039d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:20.054157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069aaee-502d-46f9-97af-2cabf3b039d9 admin admin] PATH_INFO: `/v3/users/0081f49018be43a387976158005f0552` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:20.055922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069aaee-502d-46f9-97af-2cabf3b039d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0081f49018be43a387976158005f0552)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:20.065951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069aaee-502d-46f9-97af-2cabf3b039d9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:20.067594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b82c12-e569-4bef-8148-4c024de2246b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:20.082152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b82c12-e569-4bef-8148-4c024de2246b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:20.091934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b5dd-696b-454c-b886-545980a88429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:20.094481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b5dd-696b-454c-b886-545980a88429 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:20.094481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b5dd-696b-454c-b886-545980a88429 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:20.094481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b5dd-696b-454c-b886-545980a88429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:20.117328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5b5dd-696b-454c-b886-545980a88429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:20.120640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5b5dd-696b-454c-b886-545980a88429 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:20.146283 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 813/1638] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:20.171078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d069aaee-502d-46f9-97af-2cabf3b039d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0081f49018be43a387976158005f0552'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:20.192551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:20.196363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d069aaee-502d-46f9-97af-2cabf3b039d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0081f49018be43a387976158005f0552'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:20.197840 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d069aaee-502d-46f9-97af-2cabf3b039d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0081f49018be43a387976158005f0552'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:20.219241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d069aaee-502d-46f9-97af-2cabf3b039d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0081f49018be43a387976158005f0552'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:20.220183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d069aaee-502d-46f9-97af-2cabf3b039d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0081f49018be43a387976158005f0552'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:20.258540 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 826/1639] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:58:19 2022] DELETE /identity/v3/users/0081f49018be43a387976158005f0552 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:20.262273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dfe0a-5b3d-4a77-8e54-cf458d3bc68c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:20.262614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dfe0a-5b3d-4a77-8e54-cf458d3bc68c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:20.262898 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dfe0a-5b3d-4a77-8e54-cf458d3bc68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:20.292016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf42ce-db68-4838-8d88-32a750cc14e5 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:20.292016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf42ce-db68-4838-8d88-32a750cc14e5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:20.292016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf42ce-db68-4838-8d88-32a750cc14e5 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:20.292016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 827/1640] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jun 29 09:58:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:58:20.298740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920cf91-aa4a-4475-b724-95332c030c42 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:20.299444 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920cf91-aa4a-4475-b724-95332c030c42 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:20.299873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920cf91-aa4a-4475-b724-95332c030c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:20.323539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-437dfe0a-5b3d-4a77-8e54-cf458d3bc68c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:20.324795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-437dfe0a-5b3d-4a77-8e54-cf458d3bc68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:20.332384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d920cf91-aa4a-4475-b724-95332c030c42 None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:20.333615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d920cf91-aa4a-4475-b724-95332c030c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:20.478280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 814/1641] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:20.478280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437dfe0a-5b3d-4a77-8e54-cf458d3bc68c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:20.483468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-437dfe0a-5b3d-4a77-8e54-cf458d3bc68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:20.510301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437dfe0a-5b3d-4a77-8e54-cf458d3bc68c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:20.529131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:20.530509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 828/1642] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jun 29 09:58:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:58:20.530693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:20.530693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:20.530877 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] PATH_INFO: `/v3/users/aba55f5cda1441249e414223ba8f0269` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:20.531139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920cf91-aa4a-4475-b724-95332c030c42 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:20.532981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d920cf91-aa4a-4475-b724-95332c030c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:20.534437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba55f5cda1441249e414223ba8f0269)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:20.535379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: WARNING py.warnings [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:58:20.535379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: warnings.warn(msg) Jun 29 09:58:20.535379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: Jun 29 09:58:20.536602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:20.674688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920cf91-aa4a-4475-b724-95332c030c42 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:20.676513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba55f5cda1441249e414223ba8f0269'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:20.682190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40292298-7c4d-4b94-aff2-3f8f0eb2b7d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:20.684352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292298-7c4d-4b94-aff2-3f8f0eb2b7d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:20.684724 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292298-7c4d-4b94-aff2-3f8f0eb2b7d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:20.685604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292298-7c4d-4b94-aff2-3f8f0eb2b7d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:20.714456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40292298-7c4d-4b94-aff2-3f8f0eb2b7d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:20.719298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40292298-7c4d-4b94-aff2-3f8f0eb2b7d0 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:20.734093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba55f5cda1441249e414223ba8f0269'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:20.734929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba55f5cda1441249e414223ba8f0269'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:20.738170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba55f5cda1441249e414223ba8f0269'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:20.744712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 829/1643] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:20.749483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991ba2-45b1-47d9-aaef-7261c7e8ab14 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:20.749876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991ba2-45b1-47d9-aaef-7261c7e8ab14 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:20.750228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991ba2-45b1-47d9-aaef-7261c7e8ab14 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:20.755146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 830/1644] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jun 29 09:58:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:58:20.766270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338507-419d-41ea-900e-f8b0729a6624 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:20.767134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338507-419d-41ea-900e-f8b0729a6624 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:20.767706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338507-419d-41ea-900e-f8b0729a6624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:20.770226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-69c479c1-5be4-42e6-a11c-94b5fce4f1d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba55f5cda1441249e414223ba8f0269'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:20.801282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 815/1645] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:58:20 2022] DELETE /identity/v3/users/aba55f5cda1441249e414223ba8f0269 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:20.806692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecba8b-f35c-4c46-8d7b-83d1be56b3b3 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:20.807343 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecba8b-f35c-4c46-8d7b-83d1be56b3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:20.807957 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecba8b-f35c-4c46-8d7b-83d1be56b3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:20.820429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4f338507-419d-41ea-900e-f8b0729a6624 None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:20.826222 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4f338507-419d-41ea-900e-f8b0729a6624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:20.855659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-50ecba8b-f35c-4c46-8d7b-83d1be56b3b3 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:20.859728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-50ecba8b-f35c-4c46-8d7b-83d1be56b3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:20.912329 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:20.978454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:21.023867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 831/1646] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 09:58:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:21.047285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f338507-419d-41ea-900e-f8b0729a6624 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:21.048830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4f338507-419d-41ea-900e-f8b0729a6624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:21.060199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 816/1647] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:21.087450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecba8b-f35c-4c46-8d7b-83d1be56b3b3 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:21.090781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-50ecba8b-f35c-4c46-8d7b-83d1be56b3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:21.208263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f338507-419d-41ea-900e-f8b0729a6624 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:21.230208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9829b7e-0c45-43bc-9bf2-592bbd622c32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:21.230851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829b7e-0c45-43bc-9bf2-592bbd622c32 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:21.231152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829b7e-0c45-43bc-9bf2-592bbd622c32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:21.231529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829b7e-0c45-43bc-9bf2-592bbd622c32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:21.259208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9829b7e-0c45-43bc-9bf2-592bbd622c32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:21.261050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9829b7e-0c45-43bc-9bf2-592bbd622c32 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:21.264405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecba8b-f35c-4c46-8d7b-83d1be56b3b3 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:21.270926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d90e9-517c-4ee7-841d-95981f15204b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:21.271637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d90e9-517c-4ee7-841d-95981f15204b service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:21.271856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d90e9-517c-4ee7-841d-95981f15204b service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:21.272143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d90e9-517c-4ee7-841d-95981f15204b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:21.278079 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 832/1648] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:21.326449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d90e9-517c-4ee7-841d-95981f15204b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:21.337177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d90e9-517c-4ee7-841d-95981f15204b service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:21.401575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 817/1649] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:58:23.915716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9829b7e-0c45-43bc-9bf2-592bbd622c32 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:23.917212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b9829b7e-0c45-43bc-9bf2-592bbd622c32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:23.936222 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9829b7e-0c45-43bc-9bf2-592bbd622c32 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:23.944933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:23.946006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:23.947126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:23.947584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] PATH_INFO: `/v3/projects/61d51cf573ee400ebbb92c3d68fcbefe` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:23.949143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d51cf573ee400ebbb92c3d68fcbefe)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:23.950462 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 09:58:23.950462 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jun 29 09:58:23.950462 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: Jun 29 09:58:23.952937 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:24.072742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d51cf573ee400ebbb92c3d68fcbefe', 'request_id': 'req-87b20080-02d7-40e3-b567-59d47ed11893'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:24.086675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d51cf573ee400ebbb92c3d68fcbefe', 'request_id': 'req-87b20080-02d7-40e3-b567-59d47ed11893'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:24.104067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d51cf573ee400ebbb92c3d68fcbefe', 'request_id': 'req-87b20080-02d7-40e3-b567-59d47ed11893'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:24.144733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] The token cache is being invalidate because project 61d51cf573ee400ebbb92c3d68fcbefe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:58:24.144960 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:24.147484 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:24.152436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 833/1650] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:58:23 2022] DELETE /identity/v3/projects/61d51cf573ee400ebbb92c3d68fcbefe => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:24.156736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d90e9-517c-4ee7-841d-95981f15204b service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:24.172713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d90e9-517c-4ee7-841d-95981f15204b service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:24.393105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:24.395054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:24.623057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b20080-02d7-40e3-b567-59d47ed11893 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:24.640928 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:24.641499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:24.641794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:24.642074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] PATH_INFO: `/v3/projects/d195d5e7f60544b48e25f0bedbbe8731` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:24.659410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=d195d5e7f60544b48e25f0bedbbe8731)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:24.661538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:24.765543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:24.819189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37d08e-bad0-40e7-a7a5-d2c3a7be17ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:24.820820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37d08e-bad0-40e7-a7a5-d2c3a7be17ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:24.822719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37d08e-bad0-40e7-a7a5-d2c3a7be17ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:24.822719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37d08e-bad0-40e7-a7a5-d2c3a7be17ea admin admin] PATH_INFO: `/v3/users/7aa74832ab7a4e9987e94287e990b362` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:24.824801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37d08e-bad0-40e7-a7a5-d2c3a7be17ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aa74832ab7a4e9987e94287e990b362)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:24.826826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37d08e-bad0-40e7-a7a5-d2c3a7be17ea admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:24.829041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd195d5e7f60544b48e25f0bedbbe8731', 'request_id': 'req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:24.842772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd195d5e7f60544b48e25f0bedbbe8731', 'request_id': 'req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:24.854388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd195d5e7f60544b48e25f0bedbbe8731', 'request_id': 'req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:24.899411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] The token cache is being invalidate because project d195d5e7f60544b48e25f0bedbbe8731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:58:24.899735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:24.901072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8ef3d209-ed86-43f4-9966-1d41e0d9c6ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:24.909652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 834/1651] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:58:24 2022] DELETE /identity/v3/projects/d195d5e7f60544b48e25f0bedbbe8731 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:58:24.914262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dbf5a-1168-479c-8e27-9ccba9bb316e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:24.915439 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dbf5a-1168-479c-8e27-9ccba9bb316e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:24.916121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dbf5a-1168-479c-8e27-9ccba9bb316e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:24.995622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2b37d08e-bad0-40e7-a7a5-d2c3a7be17ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aa74832ab7a4e9987e94287e990b362'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:25.045588 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2b37d08e-bad0-40e7-a7a5-d2c3a7be17ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aa74832ab7a4e9987e94287e990b362'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:25.047003 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2b37d08e-bad0-40e7-a7a5-d2c3a7be17ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aa74832ab7a4e9987e94287e990b362'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:25.047991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2b37d08e-bad0-40e7-a7a5-d2c3a7be17ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aa74832ab7a4e9987e94287e990b362'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:25.080912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:25.095658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2b37d08e-bad0-40e7-a7a5-d2c3a7be17ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aa74832ab7a4e9987e94287e990b362'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:25.146333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 818/1652] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:58:24 2022] DELETE /identity/v3/users/7aa74832ab7a4e9987e94287e990b362 => generated 0 bytes in 987 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:25.293181 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8f4dbf5a-1168-479c-8e27-9ccba9bb316e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:25.294931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8f4dbf5a-1168-479c-8e27-9ccba9bb316e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:25.415822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 835/1653] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:58:25.424105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3f5f9-e6dd-4a55-b86d-4d0550a70b35 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:25.425485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3f5f9-e6dd-4a55-b86d-4d0550a70b35 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:25.425485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3f5f9-e6dd-4a55-b86d-4d0550a70b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:25.454601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-30c3f5f9-e6dd-4a55-b86d-4d0550a70b35 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:25.461267 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-30c3f5f9-e6dd-4a55-b86d-4d0550a70b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:25.514142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 819/1654] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:25.517071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dbf5a-1168-479c-8e27-9ccba9bb316e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:25.530821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dbf5a-1168-479c-8e27-9ccba9bb316e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:25.549911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d96ec-f565-4681-b338-da89b90e0dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:25.550736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d96ec-f565-4681-b338-da89b90e0dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:25.551016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d96ec-f565-4681-b338-da89b90e0dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:25.551283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d96ec-f565-4681-b338-da89b90e0dfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:25.552077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d96ec-f565-4681-b338-da89b90e0dfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:25.552460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d96ec-f565-4681-b338-da89b90e0dfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:25.554313 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d96ec-f565-4681-b338-da89b90e0dfb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:25.580542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 836/1655] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:25.586281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3f5f9-e6dd-4a55-b86d-4d0550a70b35 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:25.640423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3f5f9-e6dd-4a55-b86d-4d0550a70b35 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:25.663267 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55bd27-105b-4567-b21d-e24344ec3aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:25.664249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55bd27-105b-4567-b21d-e24344ec3aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:25.664678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55bd27-105b-4567-b21d-e24344ec3aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:25.665098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55bd27-105b-4567-b21d-e24344ec3aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:25.665964 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55bd27-105b-4567-b21d-e24344ec3aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:25.668114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55bd27-105b-4567-b21d-e24344ec3aa3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:25.740180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 820/1656] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:25 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:25.744548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d96ec-f565-4681-b338-da89b90e0dfb admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:25.761396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d96ec-f565-4681-b338-da89b90e0dfb admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:25.927576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e8aae-f9d2-42bb-aaa3-e26decb5c845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:25.928272 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e8aae-f9d2-42bb-aaa3-e26decb5c845 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:25.928560 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e8aae-f9d2-42bb-aaa3-e26decb5c845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:25.928841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e8aae-f9d2-42bb-aaa3-e26decb5c845 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:25.929590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e8aae-f9d2-42bb-aaa3-e26decb5c845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:25.931693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e8aae-f9d2-42bb-aaa3-e26decb5c845 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:25.969788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 837/1657] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:25 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:58:25.974619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55bd27-105b-4567-b21d-e24344ec3aa3 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:25.997500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55bd27-105b-4567-b21d-e24344ec3aa3 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:26.005801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b836f5d-659a-4456-9eee-4bced8b6138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:26.007573 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b836f5d-659a-4456-9eee-4bced8b6138d admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:26.008034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b836f5d-659a-4456-9eee-4bced8b6138d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:26.008631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b836f5d-659a-4456-9eee-4bced8b6138d admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:26.009688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b836f5d-659a-4456-9eee-4bced8b6138d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:26.012046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b836f5d-659a-4456-9eee-4bced8b6138d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:26.066613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 821/1658] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:26.069513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e8aae-f9d2-42bb-aaa3-e26decb5c845 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:26.075894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b836f5d-659a-4456-9eee-4bced8b6138d admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:26.085499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e8aae-f9d2-42bb-aaa3-e26decb5c845 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:26.087151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3b836f5d-659a-4456-9eee-4bced8b6138d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:26.103568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cac6b4-31b9-4c6d-8578-c1c64dbb6326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:26.104276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cac6b4-31b9-4c6d-8578-c1c64dbb6326 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:26.104556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cac6b4-31b9-4c6d-8578-c1c64dbb6326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:26.104845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cac6b4-31b9-4c6d-8578-c1c64dbb6326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:26.105606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cac6b4-31b9-4c6d-8578-c1c64dbb6326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:26.107428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cac6b4-31b9-4c6d-8578-c1c64dbb6326 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:26.203881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 838/1659] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:26.208400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cac6b4-31b9-4c6d-8578-c1c64dbb6326 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:26.262669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cac6b4-31b9-4c6d-8578-c1c64dbb6326 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:26.277540 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e3b2d-b35c-4d54-ad37-5ca0f59672c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:26.278498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e3b2d-b35c-4d54-ad37-5ca0f59672c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:26.280798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e3b2d-b35c-4d54-ad37-5ca0f59672c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:26.283048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e3b2d-b35c-4d54-ad37-5ca0f59672c7 admin admin] PATH_INFO: `/v3/projects/490f542672b14dfdb63fdecc9b7516f6/users/9418a288f693474f8fd59223b633d58c/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:26.300225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b836f5d-659a-4456-9eee-4bced8b6138d admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:26.305412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1e8f7-8b1a-4e81-a065-26c3b357d83a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:26.306162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1e8f7-8b1a-4e81-a065-26c3b357d83a service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:26.306506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1e8f7-8b1a-4e81-a065-26c3b357d83a service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:26.306745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1e8f7-8b1a-4e81-a065-26c3b357d83a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:26.309330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-10e1e8f7-8b1a-4e81-a065-26c3b357d83a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:26.334137 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e3b2d-b35c-4d54-ad37-5ca0f59672c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=490f542672b14dfdb63fdecc9b7516f6, user_id=9418a288f693474f8fd59223b633d58c, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:26.343062 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e3b2d-b35c-4d54-ad37-5ca0f59672c7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:26.353054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:26.405488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 839/1660] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:26 2022] PUT /identity/v3/projects/490f542672b14dfdb63fdecc9b7516f6/users/9418a288f693474f8fd59223b633d58c/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:26.411933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337db045-51c7-447b-acca-2d259fd166cd None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:26.412432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337db045-51c7-447b-acca-2d259fd166cd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:26.412761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337db045-51c7-447b-acca-2d259fd166cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:26.498734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-337db045-51c7-447b-acca-2d259fd166cd None None] MFA Rules not processed for user `9418a288f693474f8fd59223b633d58c`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:26.500025 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-337db045-51c7-447b-acca-2d259fd166cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:26.509986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1e8f7-8b1a-4e81-a065-26c3b357d83a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:26.523939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1e8f7-8b1a-4e81-a065-26c3b357d83a service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:26.557309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 822/1661] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:26.677114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 840/1662] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:58:26.681696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1e8f7-8b1a-4e81-a065-26c3b357d83a service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:26.698804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1e8f7-8b1a-4e81-a065-26c3b357d83a service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:26.864244 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:26.884955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9010eca-14bf-45bc-8e78-0528100284ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:26.885995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9010eca-14bf-45bc-8e78-0528100284ad admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:26.886464 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9010eca-14bf-45bc-8e78-0528100284ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:26.886946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9010eca-14bf-45bc-8e78-0528100284ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:26.888136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9010eca-14bf-45bc-8e78-0528100284ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:26.892861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9010eca-14bf-45bc-8e78-0528100284ad admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:26.953627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 823/1663] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:26 2022] POST /identity/v3/projects => generated 371 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:26.958975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337db045-51c7-447b-acca-2d259fd166cd None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:26.973097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337db045-51c7-447b-acca-2d259fd166cd None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:27.123476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347f037-9cdd-430f-b174-3e1d5d2d7dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:27.127609 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347f037-9cdd-430f-b174-3e1d5d2d7dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:27.127609 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347f037-9cdd-430f-b174-3e1d5d2d7dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:27.127609 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347f037-9cdd-430f-b174-3e1d5d2d7dc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:27.129477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347f037-9cdd-430f-b174-3e1d5d2d7dc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:27.137708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347f037-9cdd-430f-b174-3e1d5d2d7dc2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:27.196948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9010eca-14bf-45bc-8e78-0528100284ad admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:27.200446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 841/1664] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:26 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:27.216838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9010eca-14bf-45bc-8e78-0528100284ad admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:27.229307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3aaddf-99a4-4d5e-b031-0ca48abb2ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:27.230511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aaddf-99a4-4d5e-b031-0ca48abb2ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:27.231025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aaddf-99a4-4d5e-b031-0ca48abb2ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:27.231508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aaddf-99a4-4d5e-b031-0ca48abb2ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:27.233641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3aaddf-99a4-4d5e-b031-0ca48abb2ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:27.236973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3aaddf-99a4-4d5e-b031-0ca48abb2ff0 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:27.294735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347f037-9cdd-430f-b174-3e1d5d2d7dc2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:27.306715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 824/1665] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:58:27.334783 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347f037-9cdd-430f-b174-3e1d5d2d7dc2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:27.341943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f4cab-872d-465c-991b-9d82c28a7445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:27.343961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f4cab-872d-465c-991b-9d82c28a7445 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:27.344491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f4cab-872d-465c-991b-9d82c28a7445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:27.345548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f4cab-872d-465c-991b-9d82c28a7445 admin admin] PATH_INFO: `/v3/projects/8d2b14eeac414437a66acae944fa702b/users/a8aa25eefa4f4dd88f7d90df954e6bbb/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:27.383297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f4cab-872d-465c-991b-9d82c28a7445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2b14eeac414437a66acae944fa702b, user_id=a8aa25eefa4f4dd88f7d90df954e6bbb, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:27.386786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f4cab-872d-465c-991b-9d82c28a7445 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:27.428614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 842/1666] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:27 2022] PUT /identity/v3/projects/8d2b14eeac414437a66acae944fa702b/users/a8aa25eefa4f4dd88f7d90df954e6bbb/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:27.433016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3aaddf-99a4-4d5e-b031-0ca48abb2ff0 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:27.451665 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3aaddf-99a4-4d5e-b031-0ca48abb2ff0 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:27.627489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97650c8-b87f-4212-bfb5-2aca9164361c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:27.628242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97650c8-b87f-4212-bfb5-2aca9164361c admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:27.628800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97650c8-b87f-4212-bfb5-2aca9164361c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:27.629087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97650c8-b87f-4212-bfb5-2aca9164361c admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:27.629854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97650c8-b87f-4212-bfb5-2aca9164361c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:27.631602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97650c8-b87f-4212-bfb5-2aca9164361c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:27.654445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 825/1667] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:27.659238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f4cab-872d-465c-991b-9d82c28a7445 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:27.676608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f4cab-872d-465c-991b-9d82c28a7445 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:27.692453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174b2a1-b039-4e01-a872-1975956e7293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:27.693397 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174b2a1-b039-4e01-a872-1975956e7293 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:27.693675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174b2a1-b039-4e01-a872-1975956e7293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:27.693954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174b2a1-b039-4e01-a872-1975956e7293 admin admin] PATH_INFO: `/v3/projects/8d2b14eeac414437a66acae944fa702b/users/a8aa25eefa4f4dd88f7d90df954e6bbb/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:27.696308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174b2a1-b039-4e01-a872-1975956e7293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2b14eeac414437a66acae944fa702b, user_id=a8aa25eefa4f4dd88f7d90df954e6bbb, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:27.699174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174b2a1-b039-4e01-a872-1975956e7293 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:27.750669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 843/1668] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:27 2022] PUT /identity/v3/projects/8d2b14eeac414437a66acae944fa702b/users/a8aa25eefa4f4dd88f7d90df954e6bbb/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:27.757970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880378aa-fad1-44c0-9a02-af7b8076a354 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:27.758239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880378aa-fad1-44c0-9a02-af7b8076a354 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:27.758520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880378aa-fad1-44c0-9a02-af7b8076a354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:27.839294 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-880378aa-fad1-44c0-9a02-af7b8076a354 None None] MFA Rules not processed for user `a8aa25eefa4f4dd88f7d90df954e6bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:27.843175 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-880378aa-fad1-44c0-9a02-af7b8076a354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:27.851787 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174b2a1-b039-4e01-a872-1975956e7293 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:27.853880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e174b2a1-b039-4e01-a872-1975956e7293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:27.984455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:28.018487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 826/1669] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:27 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:28.043745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880378aa-fad1-44c0-9a02-af7b8076a354 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:28.055138 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-880378aa-fad1-44c0-9a02-af7b8076a354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:28.072264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174b2a1-b039-4e01-a872-1975956e7293 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:28.089543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:28.092101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:28.092421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:28.092701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] PATH_INFO: `/v3/projects/36c4a8a1e28c4d3e96da5ec0225a1b37` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:28.096104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=36c4a8a1e28c4d3e96da5ec0225a1b37)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:28.099263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:28.173898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36c4a8a1e28c4d3e96da5ec0225a1b37', 'request_id': 'req-1f28ef48-872e-4233-bf8f-c2684eabd9fe'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:28.185981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36c4a8a1e28c4d3e96da5ec0225a1b37', 'request_id': 'req-1f28ef48-872e-4233-bf8f-c2684eabd9fe'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:28.195193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36c4a8a1e28c4d3e96da5ec0225a1b37', 'request_id': 'req-1f28ef48-872e-4233-bf8f-c2684eabd9fe'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:28.217937 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] The token cache is being invalidate because project 36c4a8a1e28c4d3e96da5ec0225a1b37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:58:28.218268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:28.220187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f28ef48-872e-4233-bf8f-c2684eabd9fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:28.222216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880378aa-fad1-44c0-9a02-af7b8076a354 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:28.222581 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 844/1670] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:58:27 2022] DELETE /identity/v3/projects/36c4a8a1e28c4d3e96da5ec0225a1b37 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:28.227809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cbb8b-b8a1-4400-92a0-45e79bedf213 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:28.228170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cbb8b-b8a1-4400-92a0-45e79bedf213 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:28.228511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cbb8b-b8a1-4400-92a0-45e79bedf213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:28.232025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:28.271196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b91cbb8b-b8a1-4400-92a0-45e79bedf213 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:28.273784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b91cbb8b-b8a1-4400-92a0-45e79bedf213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:28.335289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58dd81-c20a-4f46-ae32-5b8a516af124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:28.336038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58dd81-c20a-4f46-ae32-5b8a516af124 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:28.336325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58dd81-c20a-4f46-ae32-5b8a516af124 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:28.336600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58dd81-c20a-4f46-ae32-5b8a516af124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:28.342008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5b58dd81-c20a-4f46-ae32-5b8a516af124 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:28.371066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 845/1671] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:28.382231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e53236-c339-4ed1-8c44-766453b37426 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:28.385122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e53236-c339-4ed1-8c44-766453b37426 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:28.385407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e53236-c339-4ed1-8c44-766453b37426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:28.441902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-94e53236-c339-4ed1-8c44-766453b37426 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:28.441902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-94e53236-c339-4ed1-8c44-766453b37426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:28.462528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 846/1672] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:28.464363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0184f3a-1c4b-4fac-a091-bc3c9f3935aa None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:28.464656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0184f3a-1c4b-4fac-a091-bc3c9f3935aa None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:28.465086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0184f3a-1c4b-4fac-a091-bc3c9f3935aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:28.518130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a0184f3a-1c4b-4fac-a091-bc3c9f3935aa None None] MFA Rules not processed for user `229771b215b3492abc2c5ec38ff0d929`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:28.523673 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a0184f3a-1c4b-4fac-a091-bc3c9f3935aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:28.526218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58dd81-c20a-4f46-ae32-5b8a516af124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:28.528777 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58dd81-c20a-4f46-ae32-5b8a516af124 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:28.554611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 827/1673] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:28 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:28.555455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58dd81-c20a-4f46-ae32-5b8a516af124 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:28.579929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58dd81-c20a-4f46-ae32-5b8a516af124 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:28.585737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a89a7-7108-43be-9470-94a01c91b36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:28.586901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a89a7-7108-43be-9470-94a01c91b36f admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:28.586901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a89a7-7108-43be-9470-94a01c91b36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:28.587064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a89a7-7108-43be-9470-94a01c91b36f admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:28.588113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a89a7-7108-43be-9470-94a01c91b36f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:28.588421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a89a7-7108-43be-9470-94a01c91b36f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:28.590047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a89a7-7108-43be-9470-94a01c91b36f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:28.636019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 828/1674] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:28.636945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a89a7-7108-43be-9470-94a01c91b36f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:28.659499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a89a7-7108-43be-9470-94a01c91b36f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:28.677594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630687dc-723d-4ef7-9759-9be4fe4cf4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:28.678290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630687dc-723d-4ef7-9759-9be4fe4cf4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:28.678594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630687dc-723d-4ef7-9759-9be4fe4cf4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:28.678887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630687dc-723d-4ef7-9759-9be4fe4cf4a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:28.679617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630687dc-723d-4ef7-9759-9be4fe4cf4a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:28.681627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630687dc-723d-4ef7-9759-9be4fe4cf4a7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:28.711141 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:28.769776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 829/1675] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:28 2022] POST /identity/v3/projects => generated 363 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:28.774041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630687dc-723d-4ef7-9759-9be4fe4cf4a7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:28.782807 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-630687dc-723d-4ef7-9759-9be4fe4cf4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:28.792858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 847/1676] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:28.792858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0184f3a-1c4b-4fac-a091-bc3c9f3935aa None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:28.818637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0184f3a-1c4b-4fac-a091-bc3c9f3935aa None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:28.950018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630687dc-723d-4ef7-9759-9be4fe4cf4a7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:28.974119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d81a8d-9d2d-4c75-8f43-2a8f61e206c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:28.975247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d81a8d-9d2d-4c75-8f43-2a8f61e206c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:28.975247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d81a8d-9d2d-4c75-8f43-2a8f61e206c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:28.975382 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d81a8d-9d2d-4c75-8f43-2a8f61e206c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:28.986883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b8d81a8d-9d2d-4c75-8f43-2a8f61e206c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:28.987792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04759b-7dda-42bd-be6d-abdf22af4e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:28.988727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04759b-7dda-42bd-be6d-abdf22af4e37 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:28.989021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04759b-7dda-42bd-be6d-abdf22af4e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:28.989301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04759b-7dda-42bd-be6d-abdf22af4e37 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:28.990017 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04759b-7dda-42bd-be6d-abdf22af4e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:28.996266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04759b-7dda-42bd-be6d-abdf22af4e37 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:29.077768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 848/1677] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:28 2022] POST /identity/v3/users => generated 282 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:29.078777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04759b-7dda-42bd-be6d-abdf22af4e37 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:29.087461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2c04759b-7dda-42bd-be6d-abdf22af4e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:29.099895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:29.142358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d81a8d-9d2d-4c75-8f43-2a8f61e206c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:29.143926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d81a8d-9d2d-4c75-8f43-2a8f61e206c1 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:29.168031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 830/1678] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:29.170411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d81a8d-9d2d-4c75-8f43-2a8f61e206c1 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:29.195586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d81a8d-9d2d-4c75-8f43-2a8f61e206c1 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:29.209991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af3c3b-34b2-42ed-870a-76e9693a9248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:29.210738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af3c3b-34b2-42ed-870a-76e9693a9248 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:29.211912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af3c3b-34b2-42ed-870a-76e9693a9248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:29.212536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af3c3b-34b2-42ed-870a-76e9693a9248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:29.213664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af3c3b-34b2-42ed-870a-76e9693a9248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:29.215346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af3c3b-34b2-42ed-870a-76e9693a9248 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:29.252130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 831/1679] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:29.254231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af3c3b-34b2-42ed-870a-76e9693a9248 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:29.283685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04759b-7dda-42bd-be6d-abdf22af4e37 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:29.291401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af3c3b-34b2-42ed-870a-76e9693a9248 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:29.307085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286587f9-9453-40da-8391-55edda8997b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:29.308023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd651d4-e872-47f8-9e65-a58bb94bdfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:29.310172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286587f9-9453-40da-8391-55edda8997b3 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:29.310470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286587f9-9453-40da-8391-55edda8997b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:29.311619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd651d4-e872-47f8-9e65-a58bb94bdfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:29.312286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286587f9-9453-40da-8391-55edda8997b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:29.313086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd651d4-e872-47f8-9e65-a58bb94bdfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:29.313413 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd651d4-e872-47f8-9e65-a58bb94bdfb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:29.319883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd651d4-e872-47f8-9e65-a58bb94bdfb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:29.326631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd651d4-e872-47f8-9e65-a58bb94bdfb7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:29.378110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 832/1680] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:29.389636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd651d4-e872-47f8-9e65-a58bb94bdfb7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:29.432426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd651d4-e872-47f8-9e65-a58bb94bdfb7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:29.443949 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452ac02-eb88-4632-9126-ca36b014bcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:29.447273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452ac02-eb88-4632-9126-ca36b014bcc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:29.447640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452ac02-eb88-4632-9126-ca36b014bcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:29.449676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452ac02-eb88-4632-9126-ca36b014bcc8 admin admin] PATH_INFO: `/v3/projects/7ab2229fefa14748a4bf3a48496294d4/users/b82015fdfa894b56a0c9e533b809cd2d/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:29.491823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452ac02-eb88-4632-9126-ca36b014bcc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab2229fefa14748a4bf3a48496294d4, user_id=b82015fdfa894b56a0c9e533b809cd2d, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:29.494696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452ac02-eb88-4632-9126-ca36b014bcc8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:29.524204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 833/1681] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:29 2022] PUT /identity/v3/projects/7ab2229fefa14748a4bf3a48496294d4/users/b82015fdfa894b56a0c9e533b809cd2d/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:29.539516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f58854-f33c-433c-bc04-c17b2cee1f94 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:29.541070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f58854-f33c-433c-bc04-c17b2cee1f94 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:29.541778 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f58854-f33c-433c-bc04-c17b2cee1f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:29.558511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286587f9-9453-40da-8391-55edda8997b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:29.565578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286587f9-9453-40da-8391-55edda8997b3 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:29.607991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 849/1682] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:29.629476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-70f58854-f33c-433c-bc04-c17b2cee1f94 None None] MFA Rules not processed for user `b82015fdfa894b56a0c9e533b809cd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:29.630882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-70f58854-f33c-433c-bc04-c17b2cee1f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:29.802832 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 834/1683] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:29.809141 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286587f9-9453-40da-8391-55edda8997b3 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:29.824354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286587f9-9453-40da-8391-55edda8997b3 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:29.862075 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:30.038504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a26cb7-5544-4ee4-9ef6-b86e31b04a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:30.039222 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a26cb7-5544-4ee4-9ef6-b86e31b04a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:30.039497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a26cb7-5544-4ee4-9ef6-b86e31b04a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:30.039768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a26cb7-5544-4ee4-9ef6-b86e31b04a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:30.042516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a26cb7-5544-4ee4-9ef6-b86e31b04a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:30.044690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a26cb7-5544-4ee4-9ef6-b86e31b04a19 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:30.148062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f58854-f33c-433c-bc04-c17b2cee1f94 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:30.148751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 850/1684] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:29 2022] POST /identity/v3/projects => generated 365 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:30.170424 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f58854-f33c-433c-bc04-c17b2cee1f94 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:30.310482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580e8a5-9f85-425f-aca7-2b0885c8ff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:30.311420 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580e8a5-9f85-425f-aca7-2b0885c8ff32 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:30.311780 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580e8a5-9f85-425f-aca7-2b0885c8ff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:30.312079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580e8a5-9f85-425f-aca7-2b0885c8ff32 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:30.313122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580e8a5-9f85-425f-aca7-2b0885c8ff32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:30.318453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580e8a5-9f85-425f-aca7-2b0885c8ff32 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:30.382192 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 835/1685] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:30 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:30.386107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a26cb7-5544-4ee4-9ef6-b86e31b04a19 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:30.409226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a26cb7-5544-4ee4-9ef6-b86e31b04a19 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:30.432522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5434011-3119-42ce-b220-6074fdb0d06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:30.432522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5434011-3119-42ce-b220-6074fdb0d06f admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:30.432522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5434011-3119-42ce-b220-6074fdb0d06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:30.432994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5434011-3119-42ce-b220-6074fdb0d06f admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:30.433524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5434011-3119-42ce-b220-6074fdb0d06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:30.435238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5434011-3119-42ce-b220-6074fdb0d06f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:30.458679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 851/1686] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:58:30.463461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580e8a5-9f85-425f-aca7-2b0885c8ff32 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:30.482148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580e8a5-9f85-425f-aca7-2b0885c8ff32 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:30.495386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ec6e7-c1ab-499f-87b1-c9f622ed1081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:30.496551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ec6e7-c1ab-499f-87b1-c9f622ed1081 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:30.496921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ec6e7-c1ab-499f-87b1-c9f622ed1081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:30.497290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ec6e7-c1ab-499f-87b1-c9f622ed1081 admin admin] PATH_INFO: `/v3/projects/d72688ee562d41b6a2999708f2c60a7e/users/5f1d728c44564f69897757c6984d9244/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:30.531194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:30.549262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ec6e7-c1ab-499f-87b1-c9f622ed1081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72688ee562d41b6a2999708f2c60a7e, user_id=5f1d728c44564f69897757c6984d9244, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:30.554774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ec6e7-c1ab-499f-87b1-c9f622ed1081 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:30.575926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 836/1687] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:30 2022] PUT /identity/v3/projects/d72688ee562d41b6a2999708f2c60a7e/users/5f1d728c44564f69897757c6984d9244/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:30.580441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5434011-3119-42ce-b220-6074fdb0d06f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:30.598024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5434011-3119-42ce-b220-6074fdb0d06f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:30.732047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f8214-169f-47b7-8404-a6dd193349f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:30.736314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f8214-169f-47b7-8404-a6dd193349f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:30.737305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f8214-169f-47b7-8404-a6dd193349f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:30.738760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f8214-169f-47b7-8404-a6dd193349f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:30.743037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f8214-169f-47b7-8404-a6dd193349f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:30.746901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f8214-169f-47b7-8404-a6dd193349f2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:30.772093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ec6e7-c1ab-499f-87b1-c9f622ed1081 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:30.794638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 852/1688] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:30.806741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ec6e7-c1ab-499f-87b1-c9f622ed1081 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:30.824704 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba8674-bf0d-411e-afb1-a2783d958e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:30.825649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba8674-bf0d-411e-afb1-a2783d958e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:30.825930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba8674-bf0d-411e-afb1-a2783d958e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:30.826196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba8674-bf0d-411e-afb1-a2783d958e9d admin admin] PATH_INFO: `/v3/projects/d72688ee562d41b6a2999708f2c60a7e/users/5f1d728c44564f69897757c6984d9244/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:30.831819 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba8674-bf0d-411e-afb1-a2783d958e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72688ee562d41b6a2999708f2c60a7e, user_id=5f1d728c44564f69897757c6984d9244, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:30.834718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba8674-bf0d-411e-afb1-a2783d958e9d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:30.873877 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 837/1689] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:30 2022] PUT /identity/v3/projects/d72688ee562d41b6a2999708f2c60a7e/users/5f1d728c44564f69897757c6984d9244/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:30.887870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c0fd6-96a5-4f1f-9ea3-18bbf87c1c83 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:30.888045 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c0fd6-96a5-4f1f-9ea3-18bbf87c1c83 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:30.888333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c0fd6-96a5-4f1f-9ea3-18bbf87c1c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:30.968350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a70c0fd6-96a5-4f1f-9ea3-18bbf87c1c83 None None] MFA Rules not processed for user `5f1d728c44564f69897757c6984d9244`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:30.970250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a70c0fd6-96a5-4f1f-9ea3-18bbf87c1c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:31.214402 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 853/1690] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:31.401306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba8674-bf0d-411e-afb1-a2783d958e9d admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:31.663489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba8674-bf0d-411e-afb1-a2783d958e9d admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:31.680382 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480de66-e52d-49a8-bf0f-eaff00ffdb5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:31.683201 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480de66-e52d-49a8-bf0f-eaff00ffdb5e service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:31.684257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480de66-e52d-49a8-bf0f-eaff00ffdb5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:31.684713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480de66-e52d-49a8-bf0f-eaff00ffdb5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:31.709367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480de66-e52d-49a8-bf0f-eaff00ffdb5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:31.718552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480de66-e52d-49a8-bf0f-eaff00ffdb5e service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:31.764010 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 838/1691] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:33.351836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c0fd6-96a5-4f1f-9ea3-18bbf87c1c83 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:33.390360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:33.595499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c0fd6-96a5-4f1f-9ea3-18bbf87c1c83 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:33.605765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619fec5-527f-4f89-bde8-9cf59839875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:33.608511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619fec5-527f-4f89-bde8-9cf59839875f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:33.609074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619fec5-527f-4f89-bde8-9cf59839875f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:33.611894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619fec5-527f-4f89-bde8-9cf59839875f admin admin] PATH_INFO: `/v3/users/b82015fdfa894b56a0c9e533b809cd2d` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:33.613703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619fec5-527f-4f89-bde8-9cf59839875f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b82015fdfa894b56a0c9e533b809cd2d)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:33.618953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619fec5-527f-4f89-bde8-9cf59839875f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:33.703477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f369c82-a700-40fb-8996-21cc8a3aa5cd None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:33.704452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f369c82-a700-40fb-8996-21cc8a3aa5cd None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:33.706128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f369c82-a700-40fb-8996-21cc8a3aa5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:33.751644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e619fec5-527f-4f89-bde8-9cf59839875f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b82015fdfa894b56a0c9e533b809cd2d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:33.783953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-8f369c82-a700-40fb-8996-21cc8a3aa5cd None None] MFA Rules not processed for user `7900ba750c5f4e9faa69baaed1fd95d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:33.785486 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8f369c82-a700-40fb-8996-21cc8a3aa5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:33.801585 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e619fec5-527f-4f89-bde8-9cf59839875f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b82015fdfa894b56a0c9e533b809cd2d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:33.803542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e619fec5-527f-4f89-bde8-9cf59839875f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b82015fdfa894b56a0c9e533b809cd2d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:33.817283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e619fec5-527f-4f89-bde8-9cf59839875f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b82015fdfa894b56a0c9e533b809cd2d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:33.819070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e619fec5-527f-4f89-bde8-9cf59839875f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b82015fdfa894b56a0c9e533b809cd2d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:33.840899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 854/1692] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:58:33 2022] DELETE /identity/v3/users/b82015fdfa894b56a0c9e533b809cd2d => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:33.849191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b591-af18-4228-bc37-b87801004883 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:33.850130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b591-af18-4228-bc37-b87801004883 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:33.852426 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b591-af18-4228-bc37-b87801004883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:33.883541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8c36b591-af18-4228-bc37-b87801004883 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:33.885597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8c36b591-af18-4228-bc37-b87801004883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:33.947545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 839/1693] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:33.967693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f369c82-a700-40fb-8996-21cc8a3aa5cd None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:33.969193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8f369c82-a700-40fb-8996-21cc8a3aa5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:33.999927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 855/1694] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:34.012428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36b591-af18-4228-bc37-b87801004883 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:34.014716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8c36b591-af18-4228-bc37-b87801004883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:34.101633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:34.122520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:34.127832 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f369c82-a700-40fb-8996-21cc8a3aa5cd None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:34.137008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36b591-af18-4228-bc37-b87801004883 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:34.141596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22016da6-ea58-4a80-91d8-465e44788848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:34.142403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22016da6-ea58-4a80-91d8-465e44788848 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:34.142698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22016da6-ea58-4a80-91d8-465e44788848 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:34.142929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22016da6-ea58-4a80-91d8-465e44788848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:34.155581 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22016da6-ea58-4a80-91d8-465e44788848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:34.158081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22016da6-ea58-4a80-91d8-465e44788848 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:34.170588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 856/1695] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:58:34.172213 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9be08-6acb-41e1-95e5-320acc85681e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:34.172499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9be08-6acb-41e1-95e5-320acc85681e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:34.172781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9be08-6acb-41e1-95e5-320acc85681e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:34.204311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-87c9be08-6acb-41e1-95e5-320acc85681e None None] MFA Rules not processed for user `57605be4d856407f9ac4828ee1fef041`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:34.206463 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-87c9be08-6acb-41e1-95e5-320acc85681e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:34.288131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04259149-04bd-4dfa-b1af-9b707ff9de99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:34.288833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04259149-04bd-4dfa-b1af-9b707ff9de99 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:34.289107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04259149-04bd-4dfa-b1af-9b707ff9de99 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:34.289386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04259149-04bd-4dfa-b1af-9b707ff9de99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:34.314263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 857/1696] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:34.316686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26adb0-683b-46f7-b99f-91491cd7d36a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:34.316975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26adb0-683b-46f7-b99f-91491cd7d36a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:34.317265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26adb0-683b-46f7-b99f-91491cd7d36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:34.341045 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2f26adb0-683b-46f7-b99f-91491cd7d36a None None] MFA Rules not processed for user `57605be4d856407f9ac4828ee1fef041`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:34.342919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2f26adb0-683b-46f7-b99f-91491cd7d36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:34.357267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 858/1697] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:34.366920 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26adb0-683b-46f7-b99f-91491cd7d36a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:34.368819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2f26adb0-683b-46f7-b99f-91491cd7d36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:34.411889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26adb0-683b-46f7-b99f-91491cd7d36a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:34.431912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5dcb7-1af5-43a6-88d1-6a7e935a9149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:34.432246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5dcb7-1af5-43a6-88d1-6a7e935a9149 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:34.432533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5dcb7-1af5-43a6-88d1-6a7e935a9149 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:34.432809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5dcb7-1af5-43a6-88d1-6a7e935a9149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:34.443575 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5dcb7-1af5-43a6-88d1-6a7e935a9149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:34.445528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5dcb7-1af5-43a6-88d1-6a7e935a9149 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:34.449787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04259149-04bd-4dfa-b1af-9b707ff9de99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:34.451716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04259149-04bd-4dfa-b1af-9b707ff9de99 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:34.469177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 859/1698] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:58:34.474839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 840/1699] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:58:34.808110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5dcb7-1af5-43a6-88d1-6a7e935a9149 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:34.809698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fcf5dcb7-1af5-43a6-88d1-6a7e935a9149 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:34.830213 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5dcb7-1af5-43a6-88d1-6a7e935a9149 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:34.841182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:34.843268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:34.844496 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:34.844803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] PATH_INFO: `/v3/projects/7ab2229fefa14748a4bf3a48496294d4` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:34.847545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab2229fefa14748a4bf3a48496294d4)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:34.851173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:34.938619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab2229fefa14748a4bf3a48496294d4', 'request_id': 'req-490d0185-b73f-4d90-b0f4-1fe21016ad11'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:34.960748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab2229fefa14748a4bf3a48496294d4', 'request_id': 'req-490d0185-b73f-4d90-b0f4-1fe21016ad11'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:34.996401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab2229fefa14748a4bf3a48496294d4', 'request_id': 'req-490d0185-b73f-4d90-b0f4-1fe21016ad11'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:35.028221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] The token cache is being invalidate because project 7ab2229fefa14748a4bf3a48496294d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:58:35.029375 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:35.030356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-490d0185-b73f-4d90-b0f4-1fe21016ad11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:35.046771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04259149-04bd-4dfa-b1af-9b707ff9de99 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:35.047717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 860/1700] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:58:34 2022] DELETE /identity/v3/projects/7ab2229fefa14748a4bf3a48496294d4 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:35.063819 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04259149-04bd-4dfa-b1af-9b707ff9de99 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:35.211639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e45e4-637a-46cb-b47d-710d44dc6769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:35.213006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e45e4-637a-46cb-b47d-710d44dc6769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:35.213227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e45e4-637a-46cb-b47d-710d44dc6769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:35.213506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e45e4-637a-46cb-b47d-710d44dc6769 admin admin] PATH_INFO: `/v3/users/5f1d728c44564f69897757c6984d9244` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:35.214926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e45e4-637a-46cb-b47d-710d44dc6769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f1d728c44564f69897757c6984d9244)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:35.216905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e45e4-637a-46cb-b47d-710d44dc6769 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:35.319423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:35.322986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fa7e45e4-637a-46cb-b47d-710d44dc6769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f1d728c44564f69897757c6984d9244'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:35.353163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fa7e45e4-637a-46cb-b47d-710d44dc6769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f1d728c44564f69897757c6984d9244'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:35.354260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fa7e45e4-637a-46cb-b47d-710d44dc6769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f1d728c44564f69897757c6984d9244'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:35.355053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fa7e45e4-637a-46cb-b47d-710d44dc6769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f1d728c44564f69897757c6984d9244'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:35.367200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fa7e45e4-637a-46cb-b47d-710d44dc6769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f1d728c44564f69897757c6984d9244'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:35.397454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 841/1701] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:58:35 2022] DELETE /identity/v3/users/5f1d728c44564f69897757c6984d9244 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:58:35.496996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a5e08-120b-4003-b3aa-74d6c65b9781 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:35.497443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a5e08-120b-4003-b3aa-74d6c65b9781 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:35.498058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a5e08-120b-4003-b3aa-74d6c65b9781 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:35.499910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 861/1702] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jun 29 09:58:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:58:35.505813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f25c-c950-42be-8a68-9248a8d7ba15 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:35.506272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f25c-c950-42be-8a68-9248a8d7ba15 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:35.506744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f25c-c950-42be-8a68-9248a8d7ba15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:35.628154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-d858f25c-c950-42be-8a68-9248a8d7ba15 None None] MFA Rules not processed for user `4c97b321bf854177bd2f310cf2958959`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:35.629637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d858f25c-c950-42be-8a68-9248a8d7ba15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:35.762040 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 842/1703] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 09:58:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:35.794214 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a5e08-120b-4003-b3aa-74d6c65b9781 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:35.796464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c07a5e08-120b-4003-b3aa-74d6c65b9781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:35.826972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a5e08-120b-4003-b3aa-74d6c65b9781 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:35.842227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a98e8-8dbc-45ba-a6b7-1feeb23e0e47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:35.842955 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a98e8-8dbc-45ba-a6b7-1feeb23e0e47 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:35.843242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a98e8-8dbc-45ba-a6b7-1feeb23e0e47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:35.843729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a98e8-8dbc-45ba-a6b7-1feeb23e0e47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:35.866968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a98e8-8dbc-45ba-a6b7-1feeb23e0e47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:35.868666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a98e8-8dbc-45ba-a6b7-1feeb23e0e47 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:35.893254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 862/1704] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:37.400572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc29f20-d007-4119-8d55-28f9950c099c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:37.401496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc29f20-d007-4119-8d55-28f9950c099c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:37.401933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc29f20-d007-4119-8d55-28f9950c099c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:37.434806 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-ffc29f20-d007-4119-8d55-28f9950c099c None None] MFA Rules not processed for user `7900ba750c5f4e9faa69baaed1fd95d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:37.445349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ffc29f20-d007-4119-8d55-28f9950c099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:37.635797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 843/1705] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:37.785925 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6350fc6-4f6d-4d60-8874-d1f6d08f1821 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:37.786835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6350fc6-4f6d-4d60-8874-d1f6d08f1821 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:37.787257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6350fc6-4f6d-4d60-8874-d1f6d08f1821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:37.837807 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc29f20-d007-4119-8d55-28f9950c099c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:37.839823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ffc29f20-d007-4119-8d55-28f9950c099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:37.845042 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d6350fc6-4f6d-4d60-8874-d1f6d08f1821 None None] MFA Rules not processed for user `229771b215b3492abc2c5ec38ff0d929`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:37.846558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d6350fc6-4f6d-4d60-8874-d1f6d08f1821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:37.889503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:37.961362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc29f20-d007-4119-8d55-28f9950c099c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:37.970477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:37.973088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:37.973713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:37.974148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] PATH_INFO: `/v3/projects/d72688ee562d41b6a2999708f2c60a7e` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:37.985066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d72688ee562d41b6a2999708f2c60a7e)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:37.987821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:37.991415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:38.068437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 863/1706] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:38.070337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513244-a945-4a8c-a7d7-6406502d15e3 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:38.070959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513244-a945-4a8c-a7d7-6406502d15e3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:38.071400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513244-a945-4a8c-a7d7-6406502d15e3 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:38.073293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 864/1707] 10.222.0.52 () {54 vars in 960 bytes} [Wed Jun 29 09:58:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:58:38.079479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25838d4-c6a4-4e7a-a15f-9dda007a1194 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:38.081789 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25838d4-c6a4-4e7a-a15f-9dda007a1194 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:38.082504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25838d4-c6a4-4e7a-a15f-9dda007a1194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:38.087942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd72688ee562d41b6a2999708f2c60a7e', 'request_id': 'req-45191885-e0ac-4b31-bcc9-8181be9940b0'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:38.114537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b25838d4-c6a4-4e7a-a15f-9dda007a1194 None None] MFA Rules not processed for user `e9bff6c855ea46feaa5526d2c27c9026`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:38.115210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd72688ee562d41b6a2999708f2c60a7e', 'request_id': 'req-45191885-e0ac-4b31-bcc9-8181be9940b0'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:38.131960 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b25838d4-c6a4-4e7a-a15f-9dda007a1194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:38.147340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd72688ee562d41b6a2999708f2c60a7e', 'request_id': 'req-45191885-e0ac-4b31-bcc9-8181be9940b0'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:38.198917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] The token cache is being invalidate because project d72688ee562d41b6a2999708f2c60a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:58:38.199438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:38.206622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-45191885-e0ac-4b31-bcc9-8181be9940b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:38.208402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 844/1708] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:58:37 2022] DELETE /identity/v3/projects/d72688ee562d41b6a2999708f2c60a7e => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:58:38.211769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef99ab-9cb4-4986-a875-d11e00e700c9 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:38.211769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef99ab-9cb4-4986-a875-d11e00e700c9 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:38.211769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef99ab-9cb4-4986-a875-d11e00e700c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:38.306976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 865/1709] 10.222.0.52 () {58 vars in 1057 bytes} [Wed Jun 29 09:58:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:38.308633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c8ce4-2ea0-429f-85bb-3f7c699e079e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:38.309627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c8ce4-2ea0-429f-85bb-3f7c699e079e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:38.313837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c8ce4-2ea0-429f-85bb-3f7c699e079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:38.317661 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-41ef99ab-9cb4-4986-a875-d11e00e700c9 None None] MFA Rules not processed for user `63ba0760d13e4bbda0650de803ef46ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:38.319128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-41ef99ab-9cb4-4986-a875-d11e00e700c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:38.353797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-114c8ce4-2ea0-429f-85bb-3f7c699e079e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:38.360086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-114c8ce4-2ea0-429f-85bb-3f7c699e079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:38.553787 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 866/1710] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:38.554953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c8ce4-2ea0-429f-85bb-3f7c699e079e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:38.569595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c8ce4-2ea0-429f-85bb-3f7c699e079e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:38.574807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b630cd-f23d-4750-b087-62561d1900f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:38.575714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b630cd-f23d-4750-b087-62561d1900f3 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:38.575999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b630cd-f23d-4750-b087-62561d1900f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:38.576265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b630cd-f23d-4750-b087-62561d1900f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:38.578336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-90b630cd-f23d-4750-b087-62561d1900f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:38.619380 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 845/1711] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:38.619946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b630cd-f23d-4750-b087-62561d1900f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:38.620035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6ae86-916f-4396-8404-0d7cf5df3b04 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:38.620127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b630cd-f23d-4750-b087-62561d1900f3 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:38.620240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6ae86-916f-4396-8404-0d7cf5df3b04 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:38.620240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6ae86-916f-4396-8404-0d7cf5df3b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:38.650730 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 867/1712] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:38.655086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de502f-a796-4952-a966-3af2b4b3a542 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:38.655829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de502f-a796-4952-a966-3af2b4b3a542 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:38.656249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de502f-a796-4952-a966-3af2b4b3a542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:38.705393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-19de502f-a796-4952-a966-3af2b4b3a542 None None] MFA Rules not processed for user `63ba0760d13e4bbda0650de803ef46ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:38.707030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-19de502f-a796-4952-a966-3af2b4b3a542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:38.718453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-59f6ae86-916f-4396-8404-0d7cf5df3b04 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:38.724279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-59f6ae86-916f-4396-8404-0d7cf5df3b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:38.743603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 868/1713] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:38.764420 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 846/1714] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:38.769009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de502f-a796-4952-a966-3af2b4b3a542 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:38.792246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db1dab-8bc6-42fe-8134-97f23072a27c None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:38.792540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db1dab-8bc6-42fe-8134-97f23072a27c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:38.792914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db1dab-8bc6-42fe-8134-97f23072a27c None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:38.794767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 847/1715] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jun 29 09:58:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:58:38.800726 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de502f-a796-4952-a966-3af2b4b3a542 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:38.810925 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58a401-6729-4626-9db2-413caf291c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:38.813997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddd556-1d2b-4e57-a4a5-ebcd22f1fc9f None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:38.814415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddd556-1d2b-4e57-a4a5-ebcd22f1fc9f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:38.814969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddd556-1d2b-4e57-a4a5-ebcd22f1fc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:38.824332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58a401-6729-4626-9db2-413caf291c6b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:38.826621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58a401-6729-4626-9db2-413caf291c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:38.826946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58a401-6729-4626-9db2-413caf291c6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:38.831355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58a401-6729-4626-9db2-413caf291c6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:38.832157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58a401-6729-4626-9db2-413caf291c6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:38.838508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58a401-6729-4626-9db2-413caf291c6b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:38.868859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2dddd556-1d2b-4e57-a4a5-ebcd22f1fc9f None None] MFA Rules not processed for user `4c97b321bf854177bd2f310cf2958959`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:38.872433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2dddd556-1d2b-4e57-a4a5-ebcd22f1fc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:38.893814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 869/1716] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:38.899980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58a401-6729-4626-9db2-413caf291c6b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:38.930466 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58a401-6729-4626-9db2-413caf291c6b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:38.971884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad64d30-ee7b-4791-a9c8-ae4b44b1a74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:38.985927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad64d30-ee7b-4791-a9c8-ae4b44b1a74b admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:38.986410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad64d30-ee7b-4791-a9c8-ae4b44b1a74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:38.987798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad64d30-ee7b-4791-a9c8-ae4b44b1a74b admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:38.988261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad64d30-ee7b-4791-a9c8-ae4b44b1a74b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:38.996504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad64d30-ee7b-4791-a9c8-ae4b44b1a74b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:39.027045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 848/1717] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 09:58:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:39.038816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddd556-1d2b-4e57-a4a5-ebcd22f1fc9f None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:39.041001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2dddd556-1d2b-4e57-a4a5-ebcd22f1fc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:39.062675 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddd556-1d2b-4e57-a4a5-ebcd22f1fc9f None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:39.077124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62807b5f-6382-470a-8cf2-972d19630314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:39.077985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62807b5f-6382-470a-8cf2-972d19630314 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:39.078202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62807b5f-6382-470a-8cf2-972d19630314 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:39.078484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62807b5f-6382-470a-8cf2-972d19630314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:39.098957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:39.101183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62807b5f-6382-470a-8cf2-972d19630314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:39.102477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62807b5f-6382-470a-8cf2-972d19630314 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:39.110134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 870/1718] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:38 2022] POST /identity/v3/projects => generated 357 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:39.115032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad64d30-ee7b-4791-a9c8-ae4b44b1a74b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:39.129061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 849/1719] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:58:39.368317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad64d30-ee7b-4791-a9c8-ae4b44b1a74b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:39.384974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb4ee6-a5d1-43c0-b6ff-fafb9579dee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:39.385907 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb4ee6-a5d1-43c0-b6ff-fafb9579dee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:39.386235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb4ee6-a5d1-43c0-b6ff-fafb9579dee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:39.386647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb4ee6-a5d1-43c0-b6ff-fafb9579dee1 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:39.387482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb4ee6-a5d1-43c0-b6ff-fafb9579dee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:39.390198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb4ee6-a5d1-43c0-b6ff-fafb9579dee1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:39.442974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 871/1720] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:39 2022] POST /identity/v3/users => generated 279 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:58:39.446878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62807b5f-6382-470a-8cf2-972d19630314 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:39.476229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62807b5f-6382-470a-8cf2-972d19630314 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:39.502404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbad0d6-e8c3-44ac-9d0b-c8d4e4f3781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:39.503162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbad0d6-e8c3-44ac-9d0b-c8d4e4f3781b admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:39.503448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbad0d6-e8c3-44ac-9d0b-c8d4e4f3781b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:39.503723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbad0d6-e8c3-44ac-9d0b-c8d4e4f3781b admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:39.504472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbad0d6-e8c3-44ac-9d0b-c8d4e4f3781b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:39.506115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbad0d6-e8c3-44ac-9d0b-c8d4e4f3781b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:39.525837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:39.561478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 850/1721] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:39.563907 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb4ee6-a5d1-43c0-b6ff-fafb9579dee1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:39.578032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb4ee6-a5d1-43c0-b6ff-fafb9579dee1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:39.602420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbf766-640f-4e27-8ce7-1291da0c029b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:39.603157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf766-640f-4e27-8ce7-1291da0c029b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:39.603443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf766-640f-4e27-8ce7-1291da0c029b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:39.603723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf766-640f-4e27-8ce7-1291da0c029b admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:39.604482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbf766-640f-4e27-8ce7-1291da0c029b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:39.614837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbf766-640f-4e27-8ce7-1291da0c029b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:39.666960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbad0d6-e8c3-44ac-9d0b-c8d4e4f3781b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:39.672057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 872/1722] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:39.686237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbad0d6-e8c3-44ac-9d0b-c8d4e4f3781b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:39.718867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc4e22-7770-460f-b7fa-639682b9efa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:39.719842 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc4e22-7770-460f-b7fa-639682b9efa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:39.720123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc4e22-7770-460f-b7fa-639682b9efa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:39.720394 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc4e22-7770-460f-b7fa-639682b9efa9 admin admin] PATH_INFO: `/v3/projects/7452607201cc4220aed481577666a5b2/users/b541e5aecaeb427dbc36619730d8a92d/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:39.789470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc4e22-7770-460f-b7fa-639682b9efa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7452607201cc4220aed481577666a5b2, user_id=b541e5aecaeb427dbc36619730d8a92d, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:39.794273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc4e22-7770-460f-b7fa-639682b9efa9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:39.846521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 851/1723] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:39 2022] PUT /identity/v3/projects/7452607201cc4220aed481577666a5b2/users/b541e5aecaeb427dbc36619730d8a92d/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:58:39.857903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ecd83-17f2-46cf-ab33-84a8b587cfba None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:39.859523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ecd83-17f2-46cf-ab33-84a8b587cfba None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:39.860083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ecd83-17f2-46cf-ab33-84a8b587cfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:39.934004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-258ecd83-17f2-46cf-ab33-84a8b587cfba None None] MFA Rules not processed for user `b541e5aecaeb427dbc36619730d8a92d`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:39.937915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-258ecd83-17f2-46cf-ab33-84a8b587cfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:40.057613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc4e22-7770-460f-b7fa-639682b9efa9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:40.059651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-aefc4e22-7770-460f-b7fa-639682b9efa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:40.204958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 873/1724] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:39 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:40.211258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ecd83-17f2-46cf-ab33-84a8b587cfba None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:40.398056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc4e22-7770-460f-b7fa-639682b9efa9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:40.430533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f11da-c16f-42c1-b1ab-1c1ba305eb74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:40.442518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f11da-c16f-42c1-b1ab-1c1ba305eb74 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:40.443113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f11da-c16f-42c1-b1ab-1c1ba305eb74 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:40.443641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f11da-c16f-42c1-b1ab-1c1ba305eb74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:40.521744 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ecd83-17f2-46cf-ab33-84a8b587cfba None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:40.541190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb8fc3-fb72-481b-943b-a578ad3f9bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:40.543197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb8fc3-fb72-481b-943b-a578ad3f9bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:40.545020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb8fc3-fb72-481b-943b-a578ad3f9bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:40.545476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb8fc3-fb72-481b-943b-a578ad3f9bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:40.547653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb8fc3-fb72-481b-943b-a578ad3f9bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:40.553231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb8fc3-fb72-481b-943b-a578ad3f9bd8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:40.611039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:40.649990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f11da-c16f-42c1-b1ab-1c1ba305eb74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:40.652093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f11da-c16f-42c1-b1ab-1c1ba305eb74 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:40.666542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 874/1725] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:40 2022] POST /identity/v3/projects => generated 357 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:40.667745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb8fc3-fb72-481b-943b-a578ad3f9bd8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:40.669965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-08eb8fc3-fb72-481b-943b-a578ad3f9bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:40.685060 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 852/1726] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:58:40.688299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f11da-c16f-42c1-b1ab-1c1ba305eb74 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:40.790992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f11da-c16f-42c1-b1ab-1c1ba305eb74 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:40.982133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb8fc3-fb72-481b-943b-a578ad3f9bd8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:40.997221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4da24-e64e-4aab-8c46-42e8056d64c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:41.000509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4da24-e64e-4aab-8c46-42e8056d64c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:41.001418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4da24-e64e-4aab-8c46-42e8056d64c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:41.001418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4da24-e64e-4aab-8c46-42e8056d64c4 admin admin] PATH_INFO: `/v3/users/d081e7eeef1b4aa49f6598b9b5307c03` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:41.012712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4da24-e64e-4aab-8c46-42e8056d64c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d081e7eeef1b4aa49f6598b9b5307c03)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:41.023762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4da24-e64e-4aab-8c46-42e8056d64c4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:41.067397 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9079c6-3aa0-4ba4-bb3c-269f095f4f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:41.068679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9079c6-3aa0-4ba4-bb3c-269f095f4f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:41.068679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9079c6-3aa0-4ba4-bb3c-269f095f4f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:41.068679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9079c6-3aa0-4ba4-bb3c-269f095f4f40 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:41.069645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9079c6-3aa0-4ba4-bb3c-269f095f4f40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:41.072548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9079c6-3aa0-4ba4-bb3c-269f095f4f40 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:41.181401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 853/1727] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:40 2022] POST /identity/v3/users => generated 285 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:41.204035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9079c6-3aa0-4ba4-bb3c-269f095f4f40 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:41.209442 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-92a4da24-e64e-4aab-8c46-42e8056d64c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd081e7eeef1b4aa49f6598b9b5307c03'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:41.218390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:41.231253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9079c6-3aa0-4ba4-bb3c-269f095f4f40 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:41.245893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641deae2-229b-4138-b812-f1e91db8cef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:41.246749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641deae2-229b-4138-b812-f1e91db8cef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:41.247144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641deae2-229b-4138-b812-f1e91db8cef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:41.247513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641deae2-229b-4138-b812-f1e91db8cef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:41.248351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641deae2-229b-4138-b812-f1e91db8cef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:41.250101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641deae2-229b-4138-b812-f1e91db8cef8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:41.274126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-92a4da24-e64e-4aab-8c46-42e8056d64c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd081e7eeef1b4aa49f6598b9b5307c03'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:41.275577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-92a4da24-e64e-4aab-8c46-42e8056d64c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd081e7eeef1b4aa49f6598b9b5307c03'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:41.290754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-92a4da24-e64e-4aab-8c46-42e8056d64c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd081e7eeef1b4aa49f6598b9b5307c03'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:41.291774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-92a4da24-e64e-4aab-8c46-42e8056d64c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd081e7eeef1b4aa49f6598b9b5307c03'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:41.293081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 854/1728] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:41.294645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641deae2-229b-4138-b812-f1e91db8cef8 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:41.296278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-641deae2-229b-4138-b812-f1e91db8cef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:41.326305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641deae2-229b-4138-b812-f1e91db8cef8 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:41.333312 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 875/1729] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:58:40 2022] DELETE /identity/v3/users/d081e7eeef1b4aa49f6598b9b5307c03 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:41.339733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b171890-6538-4893-85d0-29249844feca None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:41.339920 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b171890-6538-4893-85d0-29249844feca None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:41.340376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b171890-6538-4893-85d0-29249844feca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:41.425016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8b171890-6538-4893-85d0-29249844feca None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:41.430342 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8b171890-6538-4893-85d0-29249844feca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:41.460902 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd0632-011e-4d8b-885e-2c08c36254d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:41.461970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd0632-011e-4d8b-885e-2c08c36254d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:41.462869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd0632-011e-4d8b-885e-2c08c36254d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:41.463418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd0632-011e-4d8b-885e-2c08c36254d9 admin admin] PATH_INFO: `/v3/projects/cf60e0942861430692b0dda6cdb773d6/users/70d8e5e2ab574b8b89f7f94cc32e6f9a/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:41.516041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd0632-011e-4d8b-885e-2c08c36254d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf60e0942861430692b0dda6cdb773d6, user_id=70d8e5e2ab574b8b89f7f94cc32e6f9a, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:41.535926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd0632-011e-4d8b-885e-2c08c36254d9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:41.587849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 855/1730] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:41 2022] PUT /identity/v3/projects/cf60e0942861430692b0dda6cdb773d6/users/70d8e5e2ab574b8b89f7f94cc32e6f9a/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:41.594720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd0632-011e-4d8b-885e-2c08c36254d9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:41.749587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd0632-011e-4d8b-885e-2c08c36254d9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:41.760838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b7dca-f21c-42f2-9761-105cc34890bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:41.761816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b7dca-f21c-42f2-9761-105cc34890bc admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:41.762345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b7dca-f21c-42f2-9761-105cc34890bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:41.762862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b7dca-f21c-42f2-9761-105cc34890bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:41.763821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b7dca-f21c-42f2-9761-105cc34890bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:41.765657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b7dca-f21c-42f2-9761-105cc34890bc admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:41.794668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 856/1731] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:41.799458 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b7dca-f21c-42f2-9761-105cc34890bc admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:41.824891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 876/1732] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:41.827962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b7dca-f21c-42f2-9761-105cc34890bc admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:41.841171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b171890-6538-4893-85d0-29249844feca None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:41.844212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8b171890-6538-4893-85d0-29249844feca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:41.845094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b1046-166d-43e1-ac4a-2dec4ff89df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:41.846551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b1046-166d-43e1-ac4a-2dec4ff89df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:41.847136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b1046-166d-43e1-ac4a-2dec4ff89df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:41.847611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b1046-166d-43e1-ac4a-2dec4ff89df2 admin admin] PATH_INFO: `/v3/projects/cf60e0942861430692b0dda6cdb773d6/users/70d8e5e2ab574b8b89f7f94cc32e6f9a/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:41.855612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b1046-166d-43e1-ac4a-2dec4ff89df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf60e0942861430692b0dda6cdb773d6, user_id=70d8e5e2ab574b8b89f7f94cc32e6f9a, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:41.859622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b1046-166d-43e1-ac4a-2dec4ff89df2 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:41.889962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 857/1733] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:41 2022] PUT /identity/v3/projects/cf60e0942861430692b0dda6cdb773d6/users/70d8e5e2ab574b8b89f7f94cc32e6f9a/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:41.903715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd4cd-b461-449d-ab76-0f63ab3a0847 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:41.904264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd4cd-b461-449d-ab76-0f63ab3a0847 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:41.904736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd4cd-b461-449d-ab76-0f63ab3a0847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:41.945602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:41.986942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-adebd4cd-b461-449d-ab76-0f63ab3a0847 None None] MFA Rules not processed for user `70d8e5e2ab574b8b89f7f94cc32e6f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:41.988629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-adebd4cd-b461-449d-ab76-0f63ab3a0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:42.106164 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:42.170782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b171890-6538-4893-85d0-29249844feca None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:42.234817 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 858/1734] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:41 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:42.247897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebd4cd-b461-449d-ab76-0f63ab3a0847 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:42.252176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-adebd4cd-b461-449d-ab76-0f63ab3a0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:42.340264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60237516-474f-4930-85a5-081f6f90369c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:42.340957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60237516-474f-4930-85a5-081f6f90369c service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:42.343825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60237516-474f-4930-85a5-081f6f90369c service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:42.344111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60237516-474f-4930-85a5-081f6f90369c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:42.497229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebd4cd-b461-449d-ab76-0f63ab3a0847 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:42.498323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60237516-474f-4930-85a5-081f6f90369c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:42.500203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60237516-474f-4930-85a5-081f6f90369c service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:42.512139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319ceee-54e3-448d-94b9-1274beea1b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:42.512446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 877/1735] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:42.513427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319ceee-54e3-448d-94b9-1274beea1b1c service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:42.513790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319ceee-54e3-448d-94b9-1274beea1b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:42.514073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319ceee-54e3-448d-94b9-1274beea1b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:42.552054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319ceee-54e3-448d-94b9-1274beea1b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:42.559870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319ceee-54e3-448d-94b9-1274beea1b1c service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:42.589409 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 859/1736] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:42 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:58:42.814610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60237516-474f-4930-85a5-081f6f90369c service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:42.828085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60237516-474f-4930-85a5-081f6f90369c service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:42.856631 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9da76c-5690-433c-a051-68415209bf78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:42.857469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9da76c-5690-433c-a051-68415209bf78 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:42.857855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9da76c-5690-433c-a051-68415209bf78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:42.858246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9da76c-5690-433c-a051-68415209bf78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:42.860276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5c9da76c-5690-433c-a051-68415209bf78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:42.928661 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849e114-a120-4ab7-81d2-a76689f9637d None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:42.929232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849e114-a120-4ab7-81d2-a76689f9637d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:42.929232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849e114-a120-4ab7-81d2-a76689f9637d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:42.984901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-8849e114-a120-4ab7-81d2-a76689f9637d None None] MFA Rules not processed for user `63ba0760d13e4bbda0650de803ef46ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:42.990881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8849e114-a120-4ab7-81d2-a76689f9637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:43.008622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9da76c-5690-433c-a051-68415209bf78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:43.010271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9da76c-5690-433c-a051-68415209bf78 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:43.022827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 878/1737] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:42 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:43.250105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 860/1738] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:43.449012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9da76c-5690-433c-a051-68415209bf78 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:43.450770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5c9da76c-5690-433c-a051-68415209bf78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:43.504136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9da76c-5690-433c-a051-68415209bf78 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:43.519874 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:43.522837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:43.523527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:43.523812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] PATH_INFO: `/v3/projects/f5ea4e19435c41809a46335000653767` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:43.528218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ea4e19435c41809a46335000653767)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:43.536200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:43.605686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:43.741751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ea4e19435c41809a46335000653767', 'request_id': 'req-410baba1-90d5-4d07-8697-246a58c70239'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:43.763121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ea4e19435c41809a46335000653767', 'request_id': 'req-410baba1-90d5-4d07-8697-246a58c70239'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:43.771685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ea4e19435c41809a46335000653767', 'request_id': 'req-410baba1-90d5-4d07-8697-246a58c70239'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:43.787140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] The token cache is being invalidate because project f5ea4e19435c41809a46335000653767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:58:43.787597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:43.789295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-410baba1-90d5-4d07-8697-246a58c70239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:43.795079 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 879/1739] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:58:43 2022] DELETE /identity/v3/projects/f5ea4e19435c41809a46335000653767 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:43.797441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1899-9f0c-4e9a-aa0c-d1897ba4a869 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:43.797739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1899-9f0c-4e9a-aa0c-d1897ba4a869 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:43.799924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1899-9f0c-4e9a-aa0c-d1897ba4a869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:43.874293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2f6c1899-9f0c-4e9a-aa0c-d1897ba4a869 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:43.882449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2f6c1899-9f0c-4e9a-aa0c-d1897ba4a869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:44.025439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 861/1740] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:44.033205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa5a00-84fa-4fe1-a2cf-ed04cb8a69a8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:44.033636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa5a00-84fa-4fe1-a2cf-ed04cb8a69a8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:44.034149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa5a00-84fa-4fe1-a2cf-ed04cb8a69a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:44.070013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4cfa5a00-84fa-4fe1-a2cf-ed04cb8a69a8 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:44.071329 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4cfa5a00-84fa-4fe1-a2cf-ed04cb8a69a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:44.084197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 880/1741] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:44.088933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c1899-9f0c-4e9a-aa0c-d1897ba4a869 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:44.104482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c1899-9f0c-4e9a-aa0c-d1897ba4a869 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:44.116173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5efd9-9652-4436-bf54-c574619d4394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:44.116880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5efd9-9652-4436-bf54-c574619d4394 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:44.117229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5efd9-9652-4436-bf54-c574619d4394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:44.117502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5efd9-9652-4436-bf54-c574619d4394 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:44.118242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5efd9-9652-4436-bf54-c574619d4394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:58:44.118544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5efd9-9652-4436-bf54-c574619d4394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:44.120429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5efd9-9652-4436-bf54-c574619d4394 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:44.146243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 862/1742] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:58:44.147645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa5a00-84fa-4fe1-a2cf-ed04cb8a69a8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:44.170647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa5a00-84fa-4fe1-a2cf-ed04cb8a69a8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:44.182451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148d848-79e5-4fb1-bfef-8f9fe391889a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:44.182451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148d848-79e5-4fb1-bfef-8f9fe391889a admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:44.183608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148d848-79e5-4fb1-bfef-8f9fe391889a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:44.186809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148d848-79e5-4fb1-bfef-8f9fe391889a admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:44.187803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148d848-79e5-4fb1-bfef-8f9fe391889a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:44.192435 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148d848-79e5-4fb1-bfef-8f9fe391889a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:44.243524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 881/1743] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:44 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:44.244049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5efd9-9652-4436-bf54-c574619d4394 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:44.260678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5efd9-9652-4436-bf54-c574619d4394 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:44.292313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:44.366895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c739487-5ea9-4a55-8279-9a2deb6be610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:44.368122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c739487-5ea9-4a55-8279-9a2deb6be610 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:44.368657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c739487-5ea9-4a55-8279-9a2deb6be610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:44.369219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c739487-5ea9-4a55-8279-9a2deb6be610 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:44.370420 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c739487-5ea9-4a55-8279-9a2deb6be610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:44.372588 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c739487-5ea9-4a55-8279-9a2deb6be610 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:44.427643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 863/1744] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:44 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:58:44.432239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148d848-79e5-4fb1-bfef-8f9fe391889a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:44.446932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148d848-79e5-4fb1-bfef-8f9fe391889a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:44.453462 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43df43-ec57-46ad-bb57-3668195d169c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:44.454604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43df43-ec57-46ad-bb57-3668195d169c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:44.455237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43df43-ec57-46ad-bb57-3668195d169c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:44.455603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43df43-ec57-46ad-bb57-3668195d169c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:44.456474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43df43-ec57-46ad-bb57-3668195d169c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:44.458286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43df43-ec57-46ad-bb57-3668195d169c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:44.476813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c739487-5ea9-4a55-8279-9a2deb6be610 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:44.491531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c739487-5ea9-4a55-8279-9a2deb6be610 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:44.492436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 882/1745] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:44.496876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0220911-4cc5-44a3-8ddb-7489efe9680c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:44.497643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0220911-4cc5-44a3-8ddb-7489efe9680c admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:44.497936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0220911-4cc5-44a3-8ddb-7489efe9680c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:44.498218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0220911-4cc5-44a3-8ddb-7489efe9680c admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:44.499024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0220911-4cc5-44a3-8ddb-7489efe9680c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:44.510503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0220911-4cc5-44a3-8ddb-7489efe9680c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:44.529406 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43df43-ec57-46ad-bb57-3668195d169c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:44.536842 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 864/1746] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:44.543162 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43df43-ec57-46ad-bb57-3668195d169c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:44.556870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93b1a7-eba5-4239-9bad-4a0dae5a28ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:44.559232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93b1a7-eba5-4239-9bad-4a0dae5a28ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:44.561418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93b1a7-eba5-4239-9bad-4a0dae5a28ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:44.561702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93b1a7-eba5-4239-9bad-4a0dae5a28ba admin admin] PATH_INFO: `/v3/projects/b84e6a9c58ed472dbc362eabd18a360e/users/062c56e966a8455eb5db5581edc43228/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:44.591777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93b1a7-eba5-4239-9bad-4a0dae5a28ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b84e6a9c58ed472dbc362eabd18a360e, user_id=062c56e966a8455eb5db5581edc43228, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:44.595330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93b1a7-eba5-4239-9bad-4a0dae5a28ba admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:44.634660 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 883/1747] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:44 2022] PUT /identity/v3/projects/b84e6a9c58ed472dbc362eabd18a360e/users/062c56e966a8455eb5db5581edc43228/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:58:44.649524 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47955f29-1734-4b96-908f-d7c734f6b87d None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:44.649853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47955f29-1734-4b96-908f-d7c734f6b87d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:44.650095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47955f29-1734-4b96-908f-d7c734f6b87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:44.685873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-47955f29-1734-4b96-908f-d7c734f6b87d None None] MFA Rules not processed for user `062c56e966a8455eb5db5581edc43228`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:44.687594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-47955f29-1734-4b96-908f-d7c734f6b87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:44.803687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 865/1748] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:44.808823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93b1a7-eba5-4239-9bad-4a0dae5a28ba admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:44.970613 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93b1a7-eba5-4239-9bad-4a0dae5a28ba admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:44.985584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8296c-594c-4aa5-a097-ae49ecf0a067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:44.991045 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8296c-594c-4aa5-a097-ae49ecf0a067 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:44.991609 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8296c-594c-4aa5-a097-ae49ecf0a067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:44.995497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8296c-594c-4aa5-a097-ae49ecf0a067 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:44.996280 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8296c-594c-4aa5-a097-ae49ecf0a067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:45.007361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8296c-594c-4aa5-a097-ae49ecf0a067 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:45.046758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 884/1749] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:58:44 2022] POST /identity/v3/projects => generated 381 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:45.050249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47955f29-1734-4b96-908f-d7c734f6b87d None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:45.063808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47955f29-1734-4b96-908f-d7c734f6b87d None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:45.194514 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8387576-268b-47fe-b067-e2f20c54ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:45.195717 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8387576-268b-47fe-b067-e2f20c54ca64 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:45.198233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8387576-268b-47fe-b067-e2f20c54ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:45.198867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8387576-268b-47fe-b067-e2f20c54ca64 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:45.204924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8387576-268b-47fe-b067-e2f20c54ca64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:45.209940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8387576-268b-47fe-b067-e2f20c54ca64 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:45.296564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 866/1750] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:58:45 2022] POST /identity/v3/users => generated 297 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:58:45.301110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8296c-594c-4aa5-a097-ae49ecf0a067 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:45.319195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8296c-594c-4aa5-a097-ae49ecf0a067 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:45.330285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cc409-d63c-4935-b58b-27bf80389f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:45.331501 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cc409-d63c-4935-b58b-27bf80389f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:45.333732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cc409-d63c-4935-b58b-27bf80389f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:45.334332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cc409-d63c-4935-b58b-27bf80389f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:45.336959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cc409-d63c-4935-b58b-27bf80389f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:45.341430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cc409-d63c-4935-b58b-27bf80389f5c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:45.364537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 885/1751] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:58:45.365602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8387576-268b-47fe-b067-e2f20c54ca64 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:45.380571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8387576-268b-47fe-b067-e2f20c54ca64 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:45.395260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a55aed-aa15-4380-b35d-9ee592b01212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:45.396409 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a55aed-aa15-4380-b35d-9ee592b01212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:45.397193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a55aed-aa15-4380-b35d-9ee592b01212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:45.397660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a55aed-aa15-4380-b35d-9ee592b01212 admin admin] PATH_INFO: `/v3/projects/3cda4c39a24b47aa9d199aa8a334b6b5/users/d1e5e1874d9744aeb78955f91c07948f/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:45.420811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:45.428467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a55aed-aa15-4380-b35d-9ee592b01212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cda4c39a24b47aa9d199aa8a334b6b5, user_id=d1e5e1874d9744aeb78955f91c07948f, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:45.432433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a55aed-aa15-4380-b35d-9ee592b01212 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:45.458338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 867/1752] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:45 2022] PUT /identity/v3/projects/3cda4c39a24b47aa9d199aa8a334b6b5/users/d1e5e1874d9744aeb78955f91c07948f/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:58:45.463058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cc409-d63c-4935-b58b-27bf80389f5c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:45.482269 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cc409-d63c-4935-b58b-27bf80389f5c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:45.513520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:45.604353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebae3a0-14c1-4930-bee0-e364c53b915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:45.605129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebae3a0-14c1-4930-bee0-e364c53b915a admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:45.605503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebae3a0-14c1-4930-bee0-e364c53b915a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:45.605821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebae3a0-14c1-4930-bee0-e364c53b915a admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:45.606784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebae3a0-14c1-4930-bee0-e364c53b915a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:45.609676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebae3a0-14c1-4930-bee0-e364c53b915a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:45.631961 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a55aed-aa15-4380-b35d-9ee592b01212 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:45.635160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 886/1753] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:58:45.698634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a55aed-aa15-4380-b35d-9ee592b01212 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:45.721358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed58cd8-9ebf-425d-b0e2-cf7cd1a1d52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:45.727021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed58cd8-9ebf-425d-b0e2-cf7cd1a1d52e admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:45.727469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed58cd8-9ebf-425d-b0e2-cf7cd1a1d52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:45.728071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed58cd8-9ebf-425d-b0e2-cf7cd1a1d52e admin admin] PATH_INFO: `/v3/projects/3cda4c39a24b47aa9d199aa8a334b6b5/users/d1e5e1874d9744aeb78955f91c07948f/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:45.733000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed58cd8-9ebf-425d-b0e2-cf7cd1a1d52e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cda4c39a24b47aa9d199aa8a334b6b5, user_id=d1e5e1874d9744aeb78955f91c07948f, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:45.737997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed58cd8-9ebf-425d-b0e2-cf7cd1a1d52e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:45.761539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 868/1754] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:58:45 2022] PUT /identity/v3/projects/3cda4c39a24b47aa9d199aa8a334b6b5/users/d1e5e1874d9744aeb78955f91c07948f/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:58:45.772325 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe2114-dc16-4bef-8dc0-1b4d3f2cb6e3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:45.772610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe2114-dc16-4bef-8dc0-1b4d3f2cb6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:45.772888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe2114-dc16-4bef-8dc0-1b4d3f2cb6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:45.841755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6fbe2114-dc16-4bef-8dc0-1b4d3f2cb6e3 None None] MFA Rules not processed for user `d1e5e1874d9744aeb78955f91c07948f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:45.861188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6fbe2114-dc16-4bef-8dc0-1b4d3f2cb6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:46.060511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 887/1755] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:58:46.075131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed58cd8-9ebf-425d-b0e2-cf7cd1a1d52e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:46.079328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5ed58cd8-9ebf-425d-b0e2-cf7cd1a1d52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:46.170551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed58cd8-9ebf-425d-b0e2-cf7cd1a1d52e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:46.176828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656703-7fc4-412d-b4aa-ebe28e929e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:46.177899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656703-7fc4-412d-b4aa-ebe28e929e80 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:46.177899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656703-7fc4-412d-b4aa-ebe28e929e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:46.177899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656703-7fc4-412d-b4aa-ebe28e929e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:46.216646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d656703-7fc4-412d-b4aa-ebe28e929e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:46.218789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d656703-7fc4-412d-b4aa-ebe28e929e80 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:46.252882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 869/1756] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:58:46.421608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe2114-dc16-4bef-8dc0-1b4d3f2cb6e3 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:46.435865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe2114-dc16-4bef-8dc0-1b4d3f2cb6e3 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:46.441974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812746ce-2b19-404c-a160-f3f5be77f13a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:46.443091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812746ce-2b19-404c-a160-f3f5be77f13a service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:46.443541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812746ce-2b19-404c-a160-f3f5be77f13a service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:46.444031 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812746ce-2b19-404c-a160-f3f5be77f13a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:46.614200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812746ce-2b19-404c-a160-f3f5be77f13a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:46.617679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812746ce-2b19-404c-a160-f3f5be77f13a service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:46.648174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 888/1757] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:46.941852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882efd5b-255d-45ff-8db7-31eab0c52337 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:46.942192 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882efd5b-255d-45ff-8db7-31eab0c52337 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:46.942682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882efd5b-255d-45ff-8db7-31eab0c52337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:46.981423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-882efd5b-255d-45ff-8db7-31eab0c52337 None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:46.986945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-882efd5b-255d-45ff-8db7-31eab0c52337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:47.210250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 870/1758] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 09:58:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:47.471955 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812746ce-2b19-404c-a160-f3f5be77f13a service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:47.474119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-812746ce-2b19-404c-a160-f3f5be77f13a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:47.579403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:47.691855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812746ce-2b19-404c-a160-f3f5be77f13a service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:47.713433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b228f6-b863-4c75-8c24-5ca4d145b253 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:47.715601 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b228f6-b863-4c75-8c24-5ca4d145b253 service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:47.716246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b228f6-b863-4c75-8c24-5ca4d145b253 service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:47.717324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b228f6-b863-4c75-8c24-5ca4d145b253 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:47.749565 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b228f6-b863-4c75-8c24-5ca4d145b253 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:47.757283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b228f6-b863-4c75-8c24-5ca4d145b253 service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:47.792296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 889/1759] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 09:58:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 09:58:48.716814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1608ad-7adc-4086-a582-79f14413247f None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:48.717786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1608ad-7adc-4086-a582-79f14413247f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:48.718204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1608ad-7adc-4086-a582-79f14413247f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:48.828683 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-0f1608ad-7adc-4086-a582-79f14413247f None None] MFA Rules not processed for user `4c97b321bf854177bd2f310cf2958959`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:48.830102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-0f1608ad-7adc-4086-a582-79f14413247f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:49.078929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 871/1760] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Jun 29 09:58:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:57.777493 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=18 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:58.906389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b228f6-b863-4c75-8c24-5ca4d145b253 service swift] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:59.016474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b228f6-b863-4c75-8c24-5ca4d145b253 service swift] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:59.027479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:59.028419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:59.028701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:59.028974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] PATH_INFO: `/v3/users/062c56e966a8455eb5db5581edc43228` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:59.030870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=062c56e966a8455eb5db5581edc43228)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:59.033320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:59.063663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=82,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:59.123781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '062c56e966a8455eb5db5581edc43228'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:59.143281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '062c56e966a8455eb5db5581edc43228'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:59.143966 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '062c56e966a8455eb5db5581edc43228'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:59.153039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '062c56e966a8455eb5db5581edc43228'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:59.153809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '062c56e966a8455eb5db5581edc43228'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:59.183018 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 890/1761] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:58:58 2022] DELETE /identity/v3/users/062c56e966a8455eb5db5581edc43228 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:59.186365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108323d-0c78-4bf9-a3dc-c68d169ed5d7 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:59.186685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108323d-0c78-4bf9-a3dc-c68d169ed5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:59.187019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108323d-0c78-4bf9-a3dc-c68d169ed5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:59.210909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-e108323d-0c78-4bf9-a3dc-c68d169ed5d7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:59.212992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e108323d-0c78-4bf9-a3dc-c68d169ed5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:59.306539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 872/1762] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:58:59.317615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:59.319240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:59.413142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0599642-b2a4-4660-af08-3394ca725eba admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:59.418607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea2313-965b-4b0c-90af-35c3d6d8891f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:59.418788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2313-965b-4b0c-90af-35c3d6d8891f service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:59.419074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2313-965b-4b0c-90af-35c3d6d8891f service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:59.419431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2313-965b-4b0c-90af-35c3d6d8891f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:59.433949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea2313-965b-4b0c-90af-35c3d6d8891f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:59.435908 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea2313-965b-4b0c-90af-35c3d6d8891f service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:59.447938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 891/1763] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:58:59.582888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108323d-0c78-4bf9-a3dc-c68d169ed5d7 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:59.584849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e108323d-0c78-4bf9-a3dc-c68d169ed5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:59.599447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108323d-0c78-4bf9-a3dc-c68d169ed5d7 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:59.609374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34726506-fdae-4551-8e97-ff28957917e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:59.610323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34726506-fdae-4551-8e97-ff28957917e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:59.610715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34726506-fdae-4551-8e97-ff28957917e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:59.611185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34726506-fdae-4551-8e97-ff28957917e9 admin admin] PATH_INFO: `/v3/users/54a1953b9908435592e6b38e7de9435c` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:59.612624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34726506-fdae-4551-8e97-ff28957917e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a1953b9908435592e6b38e7de9435c)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:59.614791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34726506-fdae-4551-8e97-ff28957917e9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:59.674655 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-34726506-fdae-4551-8e97-ff28957917e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a1953b9908435592e6b38e7de9435c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:59.689657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-34726506-fdae-4551-8e97-ff28957917e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a1953b9908435592e6b38e7de9435c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:59.690844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-34726506-fdae-4551-8e97-ff28957917e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a1953b9908435592e6b38e7de9435c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:59.691920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-34726506-fdae-4551-8e97-ff28957917e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a1953b9908435592e6b38e7de9435c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:59.701050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-34726506-fdae-4551-8e97-ff28957917e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a1953b9908435592e6b38e7de9435c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:59.718578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea2313-965b-4b0c-90af-35c3d6d8891f service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:59.720245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5bea2313-965b-4b0c-90af-35c3d6d8891f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:59.740254 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 873/1764] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:58:59 2022] DELETE /identity/v3/users/54a1953b9908435592e6b38e7de9435c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:58:59.742250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da100f-5e73-443d-b224-cf44ff044cff None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:59.742250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da100f-5e73-443d-b224-cf44ff044cff None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:59.742784 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da100f-5e73-443d-b224-cf44ff044cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:59.749481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea2313-965b-4b0c-90af-35c3d6d8891f service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:59.769902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:59.774929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-19da100f-5e73-443d-b224-cf44ff044cff None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:58:59.778131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-19da100f-5e73-443d-b224-cf44ff044cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:59.832156 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea2806-cf99-4cb1-97e1-8837a594898a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:59.833188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea2806-cf99-4cb1-97e1-8837a594898a service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:59.833613 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea2806-cf99-4cb1-97e1-8837a594898a service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:59.833865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea2806-cf99-4cb1-97e1-8837a594898a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:59.837745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-07ea2806-cf99-4cb1-97e1-8837a594898a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:59.855818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea2806-cf99-4cb1-97e1-8837a594898a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:59.857511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea2806-cf99-4cb1-97e1-8837a594898a service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:59.871144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 892/1765] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:58:59.878709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea2806-cf99-4cb1-97e1-8837a594898a service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:59.881020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-07ea2806-cf99-4cb1-97e1-8837a594898a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:59.889696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 874/1766] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:58:59.907026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea2806-cf99-4cb1-97e1-8837a594898a service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:59.912669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:59.913168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da100f-5e73-443d-b224-cf44ff044cff None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:58:59.913706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:59.914173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:59.914651 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] PATH_INFO: `/v3/projects/b84e6a9c58ed472dbc362eabd18a360e` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:59.916170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b84e6a9c58ed472dbc362eabd18a360e)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:59.920303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:59.931558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-19da100f-5e73-443d-b224-cf44ff044cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:58:59.954304 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da100f-5e73-443d-b224-cf44ff044cff None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:58:59.959856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9310dd-a0e9-4c77-ad86-1c07491e9722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:58:59.960701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9310dd-a0e9-4c77-ad86-1c07491e9722 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:58:59.961080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9310dd-a0e9-4c77-ad86-1c07491e9722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:58:59.961503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9310dd-a0e9-4c77-ad86-1c07491e9722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:58:59.975220 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9310dd-a0e9-4c77-ad86-1c07491e9722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:58:59.979123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9310dd-a0e9-4c77-ad86-1c07491e9722 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:58:59.988079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:58:59.994519 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b84e6a9c58ed472dbc362eabd18a360e', 'request_id': 'req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:58:59.996386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 875/1767] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:58:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:00.004551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b84e6a9c58ed472dbc362eabd18a360e', 'request_id': 'req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:00.010925 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b84e6a9c58ed472dbc362eabd18a360e', 'request_id': 'req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:00.023607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] The token cache is being invalidate because project b84e6a9c58ed472dbc362eabd18a360e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:00.023894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:00.024720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:00.026346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 893/1768] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:58:59 2022] DELETE /identity/v3/projects/b84e6a9c58ed472dbc362eabd18a360e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:00.030877 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9310dd-a0e9-4c77-ad86-1c07491e9722 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:00.047978 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9310dd-a0e9-4c77-ad86-1c07491e9722 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:00.141186 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:00.141910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:00.141910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:00.142707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] PATH_INFO: `/v3/users/d1e5e1874d9744aeb78955f91c07948f` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:00.143883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1e5e1874d9744aeb78955f91c07948f)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:00.146086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:00.206359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1e5e1874d9744aeb78955f91c07948f'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:00.221639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1e5e1874d9744aeb78955f91c07948f'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:00.222709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1e5e1874d9744aeb78955f91c07948f'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:00.223336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1e5e1874d9744aeb78955f91c07948f'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:00.232039 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1e5e1874d9744aeb78955f91c07948f'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:00.252440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 876/1769] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:00 2022] DELETE /identity/v3/users/d1e5e1874d9744aeb78955f91c07948f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:01.910301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:01.911582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:01.935716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f942b29-3745-4ce7-9d23-a9d2e488f5e6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:02.015756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:02.016891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:02.017382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:02.017833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] PATH_INFO: `/v3/projects/c1f89948b1e9480583555977ac15a6e0` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:02.019132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f89948b1e9480583555977ac15a6e0)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:02.021265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:02.085779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f89948b1e9480583555977ac15a6e0', 'request_id': 'req-6289e470-8fbb-4f3a-9b91-7ec571502350'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:02.088936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:02.116884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f89948b1e9480583555977ac15a6e0', 'request_id': 'req-6289e470-8fbb-4f3a-9b91-7ec571502350'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:02.135229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f89948b1e9480583555977ac15a6e0', 'request_id': 'req-6289e470-8fbb-4f3a-9b91-7ec571502350'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:02.161554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] The token cache is being invalidate because project c1f89948b1e9480583555977ac15a6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:02.161554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:02.161974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6289e470-8fbb-4f3a-9b91-7ec571502350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:02.164127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 894/1770] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:01 2022] DELETE /identity/v3/projects/c1f89948b1e9480583555977ac15a6e0 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:02.168538 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:02.170735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:02.200766 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61f708-6197-43f3-8fe2-0662ba395714 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:02.202965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a9b7a-1d92-4696-81c8-1c314a7c8569 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:02.203250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a9b7a-1d92-4696-81c8-1c314a7c8569 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:02.203522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a9b7a-1d92-4696-81c8-1c314a7c8569 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:02.205410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 895/1771] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 09:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:59:02.302428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c403c-e2ce-450d-8cef-315b78649167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:02.304063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c403c-e2ce-450d-8cef-315b78649167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:02.304358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c403c-e2ce-450d-8cef-315b78649167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:02.304639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c403c-e2ce-450d-8cef-315b78649167 admin admin] PATH_INFO: `/v3/users/cc4fd9d5664547e6950f468489257f50` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:02.305953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c403c-e2ce-450d-8cef-315b78649167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc4fd9d5664547e6950f468489257f50)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:02.308905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c403c-e2ce-450d-8cef-315b78649167 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:02.362006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:02.364584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59cc72-5d04-4ec6-961d-3284c53b9914 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:02.364901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59cc72-5d04-4ec6-961d-3284c53b9914 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:02.365911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59cc72-5d04-4ec6-961d-3284c53b9914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:02.411552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c21c403c-e2ce-450d-8cef-315b78649167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc4fd9d5664547e6950f468489257f50'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:02.428252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-da59cc72-5d04-4ec6-961d-3284c53b9914 None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:02.429860 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-da59cc72-5d04-4ec6-961d-3284c53b9914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:02.459590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c21c403c-e2ce-450d-8cef-315b78649167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc4fd9d5664547e6950f468489257f50'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:02.460599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c21c403c-e2ce-450d-8cef-315b78649167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc4fd9d5664547e6950f468489257f50'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:02.461361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c21c403c-e2ce-450d-8cef-315b78649167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc4fd9d5664547e6950f468489257f50'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:02.477402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c21c403c-e2ce-450d-8cef-315b78649167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc4fd9d5664547e6950f468489257f50'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:02.499091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 877/1772] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:02 2022] DELETE /identity/v3/users/cc4fd9d5664547e6950f468489257f50 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:02.574019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f797567-d52a-4632-882f-8de78c0c76df None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:02.575285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f797567-d52a-4632-882f-8de78c0c76df None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:02.575699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f797567-d52a-4632-882f-8de78c0c76df None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:02.579442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 878/1773] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 09:59:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 09:59:02.627269 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 896/1774] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 09:59:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:02.692142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906652ee-4c30-47b5-be70-62212c640e94 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:02.692507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906652ee-4c30-47b5-be70-62212c640e94 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:02.692861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906652ee-4c30-47b5-be70-62212c640e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:02.729863 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-906652ee-4c30-47b5-be70-62212c640e94 None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:02.731280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-906652ee-4c30-47b5-be70-62212c640e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:02.769996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59cc72-5d04-4ec6-961d-3284c53b9914 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:02.771549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-da59cc72-5d04-4ec6-961d-3284c53b9914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:02.805002 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59cc72-5d04-4ec6-961d-3284c53b9914 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:02.946933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 879/1775] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 09:59:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:02.957735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:02.959285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:02.959915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:02.960397 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] PATH_INFO: `/v3/projects/3cda4c39a24b47aa9d199aa8a334b6b5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:02.962451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cda4c39a24b47aa9d199aa8a334b6b5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:02.964852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:03.042921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cda4c39a24b47aa9d199aa8a334b6b5', 'request_id': 'req-9b6ac478-4660-4590-9d7f-c006ddd19915'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:03.064171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cda4c39a24b47aa9d199aa8a334b6b5', 'request_id': 'req-9b6ac478-4660-4590-9d7f-c006ddd19915'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:03.071941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cda4c39a24b47aa9d199aa8a334b6b5', 'request_id': 'req-9b6ac478-4660-4590-9d7f-c006ddd19915'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:03.097743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:03.107772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] The token cache is being invalidate because project 3cda4c39a24b47aa9d199aa8a334b6b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:03.108914 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:03.110684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:03.116722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 897/1776] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:02 2022] DELETE /identity/v3/projects/3cda4c39a24b47aa9d199aa8a334b6b5 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:03.119806 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605feda4-d533-4adf-bbfd-3aaa74fc1506 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:03.120280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605feda4-d533-4adf-bbfd-3aaa74fc1506 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:03.120696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605feda4-d533-4adf-bbfd-3aaa74fc1506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:03.166637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-605feda4-d533-4adf-bbfd-3aaa74fc1506 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:03.167856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-605feda4-d533-4adf-bbfd-3aaa74fc1506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:03.176119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:03.177825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:03.316938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 880/1777] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:03.318991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45954deb-f483-4276-861f-01305983ffd8 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:03.319565 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45954deb-f483-4276-861f-01305983ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:03.319848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45954deb-f483-4276-861f-01305983ffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:03.368047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-45954deb-f483-4276-861f-01305983ffd8 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:03.369297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-45954deb-f483-4276-861f-01305983ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:03.376208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ac478-4660-4590-9d7f-c006ddd19915 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:03.386068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdb35b-8d0f-47a7-9e15-73a242f5ce05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:03.387101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdb35b-8d0f-47a7-9e15-73a242f5ce05 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:03.388008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdb35b-8d0f-47a7-9e15-73a242f5ce05 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:03.388503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdb35b-8d0f-47a7-9e15-73a242f5ce05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:03.399351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 881/1778] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:03.400063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-57fdb35b-8d0f-47a7-9e15-73a242f5ce05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:03.400876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45954deb-f483-4276-861f-01305983ffd8 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:03.420592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45954deb-f483-4276-861f-01305983ffd8 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:03.435497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdb35b-8d0f-47a7-9e15-73a242f5ce05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:03.437638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780cc7b-156b-438e-a841-101f4191715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:03.438386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780cc7b-156b-438e-a841-101f4191715f admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:03.438733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780cc7b-156b-438e-a841-101f4191715f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:03.439150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780cc7b-156b-438e-a841-101f4191715f admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:03.440312 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780cc7b-156b-438e-a841-101f4191715f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:03.440623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780cc7b-156b-438e-a841-101f4191715f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:03.442261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780cc7b-156b-438e-a841-101f4191715f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:03.445415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdb35b-8d0f-47a7-9e15-73a242f5ce05 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:03.479342 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 898/1779] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:03.489711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdb35b-8d0f-47a7-9e15-73a242f5ce05 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:03.491726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 882/1780] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:03.523082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdb35b-8d0f-47a7-9e15-73a242f5ce05 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:03.530746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db591ce3-b721-4f78-9152-e75c2d6ca7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:03.531622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591ce3-b721-4f78-9152-e75c2d6ca7c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:03.532101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591ce3-b721-4f78-9152-e75c2d6ca7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:03.532522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591ce3-b721-4f78-9152-e75c2d6ca7c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:03.533903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db591ce3-b721-4f78-9152-e75c2d6ca7c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:03.547464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db591ce3-b721-4f78-9152-e75c2d6ca7c1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:03.616980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 899/1781] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:03 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:03.619459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780cc7b-156b-438e-a841-101f4191715f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:03.635049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780cc7b-156b-438e-a841-101f4191715f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:03.645142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:03.711346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db591ce3-b721-4f78-9152-e75c2d6ca7c1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:03.726770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca13c5-9305-4388-8055-2acad877461f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:03.727484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca13c5-9305-4388-8055-2acad877461f admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:03.727770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca13c5-9305-4388-8055-2acad877461f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:03.728771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca13c5-9305-4388-8055-2acad877461f admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:03.729610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca13c5-9305-4388-8055-2acad877461f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:03.731855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca13c5-9305-4388-8055-2acad877461f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:03.763278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 883/1782] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:03 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:03.769846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca13c5-9305-4388-8055-2acad877461f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:03.783924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca13c5-9305-4388-8055-2acad877461f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:03.790919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af8709-ab80-446c-82bc-5a4d7aa725a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:03.792011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af8709-ab80-446c-82bc-5a4d7aa725a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:03.792621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af8709-ab80-446c-82bc-5a4d7aa725a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:03.793221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af8709-ab80-446c-82bc-5a4d7aa725a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:03.794201 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af8709-ab80-446c-82bc-5a4d7aa725a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:03.796051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af8709-ab80-446c-82bc-5a4d7aa725a0 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:03.811748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 884/1783] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:03.817768 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af8709-ab80-446c-82bc-5a4d7aa725a0 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:03.840136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af8709-ab80-446c-82bc-5a4d7aa725a0 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:03.848544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113503cc-731d-4616-b1d4-2ed081c714d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:03.849483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113503cc-731d-4616-b1d4-2ed081c714d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:03.849929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113503cc-731d-4616-b1d4-2ed081c714d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:03.850351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113503cc-731d-4616-b1d4-2ed081c714d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:03.851303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113503cc-731d-4616-b1d4-2ed081c714d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:03.853626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113503cc-731d-4616-b1d4-2ed081c714d9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:03.869259 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 885/1784] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:03.873365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113503cc-731d-4616-b1d4-2ed081c714d9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:03.881619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db591ce3-b721-4f78-9152-e75c2d6ca7c1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:03.901604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad133e-f665-4f4d-8f5a-b5d0b76a8dc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:03.904454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad133e-f665-4f4d-8f5a-b5d0b76a8dc1 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:03.906092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad133e-f665-4f4d-8f5a-b5d0b76a8dc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:03.906522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad133e-f665-4f4d-8f5a-b5d0b76a8dc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:03.908334 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113503cc-731d-4616-b1d4-2ed081c714d9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:03.911280 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-73ad133e-f665-4f4d-8f5a-b5d0b76a8dc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:03.922756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81e342-63a4-4263-9f12-50541d053676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:03.925065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81e342-63a4-4263-9f12-50541d053676 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:03.925446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81e342-63a4-4263-9f12-50541d053676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:03.928438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81e342-63a4-4263-9f12-50541d053676 admin admin] PATH_INFO: `/v3/projects/c4ce1b344f7b4b86a42016390775f468/users/74d87b5ce40c42a38a59cd5194ca77aa/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:03.946483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad133e-f665-4f4d-8f5a-b5d0b76a8dc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:03.949280 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad133e-f665-4f4d-8f5a-b5d0b76a8dc1 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:03.973483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81e342-63a4-4263-9f12-50541d053676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ce1b344f7b4b86a42016390775f468, user_id=74d87b5ce40c42a38a59cd5194ca77aa, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:03.977092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81e342-63a4-4263-9f12-50541d053676 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:03.987603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 900/1785] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:04.011422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 886/1786] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:03 2022] PUT /identity/v3/projects/c4ce1b344f7b4b86a42016390775f468/users/74d87b5ce40c42a38a59cd5194ca77aa/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:04.017244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af89c4-ba75-4578-972e-b3584ece4785 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.017542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af89c4-ba75-4578-972e-b3584ece4785 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.017822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af89c4-ba75-4578-972e-b3584ece4785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.099258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e1af89c4-ba75-4578-972e-b3584ece4785 None None] MFA Rules not processed for user `74d87b5ce40c42a38a59cd5194ca77aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:04.100654 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e1af89c4-ba75-4578-972e-b3584ece4785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:04.170270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:04.246380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 901/1787] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:04.250950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81e342-63a4-4263-9f12-50541d053676 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:04.339378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81e342-63a4-4263-9f12-50541d053676 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:04.344633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67744813-ee65-4bc8-9f87-1fccc3cc9cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:04.345430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67744813-ee65-4bc8-9f87-1fccc3cc9cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.345714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67744813-ee65-4bc8-9f87-1fccc3cc9cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.345991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67744813-ee65-4bc8-9f87-1fccc3cc9cd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.346761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67744813-ee65-4bc8-9f87-1fccc3cc9cd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:04.348735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67744813-ee65-4bc8-9f87-1fccc3cc9cd8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:04.386856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 887/1788] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:04 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:04.390987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af89c4-ba75-4578-972e-b3584ece4785 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:04.409454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af89c4-ba75-4578-972e-b3584ece4785 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:04.411160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67744813-ee65-4bc8-9f87-1fccc3cc9cd8 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:04.413316 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-67744813-ee65-4bc8-9f87-1fccc3cc9cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:04.470048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:04.479474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef6d59-5785-4a2b-80f3-0ef49e016003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:04.480163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef6d59-5785-4a2b-80f3-0ef49e016003 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.480444 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef6d59-5785-4a2b-80f3-0ef49e016003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.480730 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef6d59-5785-4a2b-80f3-0ef49e016003 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.481433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef6d59-5785-4a2b-80f3-0ef49e016003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:04.483997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef6d59-5785-4a2b-80f3-0ef49e016003 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:04.492918 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67744813-ee65-4bc8-9f87-1fccc3cc9cd8 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:04.498374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:04.499288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.499613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.499890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] PATH_INFO: `/v3/projects/c1ce60b7a32544f2b60da5efbe0d1e85` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.501081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ce60b7a32544f2b60da5efbe0d1e85)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:04.503241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:04.508483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 902/1789] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:04 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:04.509744 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef6d59-5785-4a2b-80f3-0ef49e016003 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:04.519761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef6d59-5785-4a2b-80f3-0ef49e016003 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:04.524337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110d620-84d5-4d46-8f30-928c48f33484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:04.524996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110d620-84d5-4d46-8f30-928c48f33484 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.525301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110d620-84d5-4d46-8f30-928c48f33484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.525960 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110d620-84d5-4d46-8f30-928c48f33484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.526264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110d620-84d5-4d46-8f30-928c48f33484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:04.527977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110d620-84d5-4d46-8f30-928c48f33484 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:04.542228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 903/1790] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:04.543366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110d620-84d5-4d46-8f30-928c48f33484 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:04.552564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ce60b7a32544f2b60da5efbe0d1e85', 'request_id': 'req-61bef2c0-1670-41d7-a29b-ad2a46062ad4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:04.553501 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110d620-84d5-4d46-8f30-928c48f33484 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:04.562034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ce60b7a32544f2b60da5efbe0d1e85', 'request_id': 'req-61bef2c0-1670-41d7-a29b-ad2a46062ad4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:04.569747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ce60b7a32544f2b60da5efbe0d1e85', 'request_id': 'req-61bef2c0-1670-41d7-a29b-ad2a46062ad4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:04.583802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] The token cache is being invalidate because project c1ce60b7a32544f2b60da5efbe0d1e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:04.584192 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:04.585087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-61bef2c0-1670-41d7-a29b-ad2a46062ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:04.590947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 888/1791] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:04 2022] DELETE /identity/v3/projects/c1ce60b7a32544f2b60da5efbe0d1e85 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:04.592962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fc0d9-5de6-4481-88e0-f1cc38c2ac0f None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.593440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fc0d9-5de6-4481-88e0-f1cc38c2ac0f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.593775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fc0d9-5de6-4481-88e0-f1cc38c2ac0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.614274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fd4fc0d9-5de6-4481-88e0-f1cc38c2ac0f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:04.615585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fd4fc0d9-5de6-4481-88e0-f1cc38c2ac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:04.639350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a0e7a-2347-4450-8c16-dfd46a5a6cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:04.640266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a0e7a-2347-4450-8c16-dfd46a5a6cad admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.640455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a0e7a-2347-4450-8c16-dfd46a5a6cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.640741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a0e7a-2347-4450-8c16-dfd46a5a6cad admin admin] PATH_INFO: `/v3/projects/6daaf7e5026441759a0a7b2d663fa92f/users/f91fc14906f94aabb21b3cc9e36a7c21/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.662799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a0e7a-2347-4450-8c16-dfd46a5a6cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=6daaf7e5026441759a0a7b2d663fa92f, user_id=f91fc14906f94aabb21b3cc9e36a7c21, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:04.665667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a0e7a-2347-4450-8c16-dfd46a5a6cad admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:04.685051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 904/1792] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:04 2022] PUT /identity/v3/projects/6daaf7e5026441759a0a7b2d663fa92f/users/f91fc14906f94aabb21b3cc9e36a7c21/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:04.688742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a0e7a-2347-4450-8c16-dfd46a5a6cad admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:04.694739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 889/1793] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:04.698816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c33a0e7a-2347-4450-8c16-dfd46a5a6cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:04.703218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc7cee-2872-4633-b468-6ff772c75fcf None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.703505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc7cee-2872-4633-b468-6ff772c75fcf None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.703802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc7cee-2872-4633-b468-6ff772c75fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.731965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-f8bc7cee-2872-4633-b468-6ff772c75fcf None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:04.742443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f8bc7cee-2872-4633-b468-6ff772c75fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:04.757454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:04.779307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a0e7a-2347-4450-8c16-dfd46a5a6cad admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:04.784637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb31e5d-5a6e-411e-b0ff-cdb2da2345c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:04.785316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb31e5d-5a6e-411e-b0ff-cdb2da2345c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.785593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb31e5d-5a6e-411e-b0ff-cdb2da2345c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.785860 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb31e5d-5a6e-411e-b0ff-cdb2da2345c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.786711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb31e5d-5a6e-411e-b0ff-cdb2da2345c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:04.788300 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb31e5d-5a6e-411e-b0ff-cdb2da2345c5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:04.799412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 890/1794] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:04.799614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 905/1795] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:04.803421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb31e5d-5a6e-411e-b0ff-cdb2da2345c5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:04.803586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc7cee-2872-4633-b468-6ff772c75fcf None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:04.813234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc7cee-2872-4633-b468-6ff772c75fcf None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:04.813948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb31e5d-5a6e-411e-b0ff-cdb2da2345c5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:04.818674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581a2e-6c22-4e0f-bd56-053883728af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:04.819570 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581a2e-6c22-4e0f-bd56-053883728af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.819843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581a2e-6c22-4e0f-bd56-053883728af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.820108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581a2e-6c22-4e0f-bd56-053883728af5 admin admin] PATH_INFO: `/v3/projects/6daaf7e5026441759a0a7b2d663fa92f/users/f91fc14906f94aabb21b3cc9e36a7c21/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.820427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf28fa-a726-4825-ade7-e2d7f186bec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:04.821130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf28fa-a726-4825-ade7-e2d7f186bec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.821416 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf28fa-a726-4825-ade7-e2d7f186bec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.821689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf28fa-a726-4825-ade7-e2d7f186bec7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.822057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4581a2e-6c22-4e0f-bd56-053883728af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6daaf7e5026441759a0a7b2d663fa92f, user_id=f91fc14906f94aabb21b3cc9e36a7c21, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:04.822427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf28fa-a726-4825-ade7-e2d7f186bec7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:04.822769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf28fa-a726-4825-ade7-e2d7f186bec7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:04.824392 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf28fa-a726-4825-ade7-e2d7f186bec7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:04.824732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4581a2e-6c22-4e0f-bd56-053883728af5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:04.831995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:04.835915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 906/1796] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:04 2022] PUT /identity/v3/projects/6daaf7e5026441759a0a7b2d663fa92f/users/f91fc14906f94aabb21b3cc9e36a7c21/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:04.841463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 891/1797] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:04.845687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581a2e-6c22-4e0f-bd56-053883728af5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:04.846391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beb81c-c5f3-4c30-ac9b-8539a74dce33 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.846833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beb81c-c5f3-4c30-ac9b-8539a74dce33 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.847304 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beb81c-c5f3-4c30-ac9b-8539a74dce33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.874549 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b4beb81c-c5f3-4c30-ac9b-8539a74dce33 None None] MFA Rules not processed for user `f91fc14906f94aabb21b3cc9e36a7c21`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:04.875894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b4beb81c-c5f3-4c30-ac9b-8539a74dce33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:04.907117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581a2e-6c22-4e0f-bd56-053883728af5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:04.912278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1a792-532a-4093-9990-c50b339f32f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:04.913099 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1a792-532a-4093-9990-c50b339f32f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:04.913380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1a792-532a-4093-9990-c50b339f32f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:04.913650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1a792-532a-4093-9990-c50b339f32f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:04.914366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1a792-532a-4093-9990-c50b339f32f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:04.916369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1a792-532a-4093-9990-c50b339f32f6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:04.944403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 907/1798] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:04 2022] POST /identity/v3/projects => generated 393 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:04.948714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1a792-532a-4093-9990-c50b339f32f6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:04.972964 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 892/1799] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:04.984684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4beb81c-c5f3-4c30-ac9b-8539a74dce33 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:04.986332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b4beb81c-c5f3-4c30-ac9b-8539a74dce33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:05.013296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1a792-532a-4093-9990-c50b339f32f6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.018210 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b62880-026f-41ff-9761-51329ef265a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.019048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b62880-026f-41ff-9761-51329ef265a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.019399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b62880-026f-41ff-9761-51329ef265a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.019741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b62880-026f-41ff-9761-51329ef265a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.020535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b62880-026f-41ff-9761-51329ef265a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.022550 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b62880-026f-41ff-9761-51329ef265a9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.043416 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 908/1800] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:04 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:05.043624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4beb81c-c5f3-4c30-ac9b-8539a74dce33 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.047317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b62880-026f-41ff-9761-51329ef265a9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:05.057599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b62880-026f-41ff-9761-51329ef265a9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.062523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbea26-9613-49d7-a12a-63ad09510e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.063237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbea26-9613-49d7-a12a-63ad09510e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.063514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbea26-9613-49d7-a12a-63ad09510e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.063783 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbea26-9613-49d7-a12a-63ad09510e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.064707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dbea26-9613-49d7-a12a-63ad09510e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.066322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dbea26-9613-49d7-a12a-63ad09510e8c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.076654 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 909/1801] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:05.081101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbea26-9613-49d7-a12a-63ad09510e8c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:05.091487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbea26-9613-49d7-a12a-63ad09510e8c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.096367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76291-116e-45d0-b0a6-60b56cb50224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.097035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76291-116e-45d0-b0a6-60b56cb50224 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.097330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76291-116e-45d0-b0a6-60b56cb50224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.097592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76291-116e-45d0-b0a6-60b56cb50224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.097712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0c05e-ac5d-41c5-96a6-8f08f82840c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.098302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c05e-ac5d-41c5-96a6-8f08f82840c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.098429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb76291-116e-45d0-b0a6-60b56cb50224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.098578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c05e-ac5d-41c5-96a6-8f08f82840c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.098858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c05e-ac5d-41c5-96a6-8f08f82840c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.099956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb76291-116e-45d0-b0a6-60b56cb50224 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.114627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 910/1802] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:05.115622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76291-116e-45d0-b0a6-60b56cb50224 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:05.126147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76291-116e-45d0-b0a6-60b56cb50224 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.131170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967efc2-5193-4b12-9077-72536f65908f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.132070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967efc2-5193-4b12-9077-72536f65908f admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.132350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967efc2-5193-4b12-9077-72536f65908f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.132627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967efc2-5193-4b12-9077-72536f65908f admin admin] PATH_INFO: `/v3/projects/506065ed60c141e7ab316b5892b41b77/users/c9509e42085b4fa5a5ba6a2ef28e7efd/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.145308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:05.148933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967efc2-5193-4b12-9077-72536f65908f admin admin] RBAC: Authorizing `identity:create_grant(project_id=506065ed60c141e7ab316b5892b41b77, user_id=c9509e42085b4fa5a5ba6a2ef28e7efd, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.150191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:05.152285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967efc2-5193-4b12-9077-72536f65908f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.162746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0c05e-ac5d-41c5-96a6-8f08f82840c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.164125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0c05e-ac5d-41c5-96a6-8f08f82840c7 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.167152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 911/1803] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:05 2022] PUT /identity/v3/projects/506065ed60c141e7ab316b5892b41b77/users/c9509e42085b4fa5a5ba6a2ef28e7efd/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:05.175028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 893/1804] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:05.177421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8b9d-de1a-46f8-91b7-3af88332328f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.177707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8b9d-de1a-46f8-91b7-3af88332328f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.178190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8b9d-de1a-46f8-91b7-3af88332328f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.206426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0c05e-ac5d-41c5-96a6-8f08f82840c7 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:05.208493 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a95a8b9d-de1a-46f8-91b7-3af88332328f None None] MFA Rules not processed for user `c9509e42085b4fa5a5ba6a2ef28e7efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:05.210888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a95a8b9d-de1a-46f8-91b7-3af88332328f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:05.218368 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0c05e-ac5d-41c5-96a6-8f08f82840c7 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.293388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 912/1805] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:05.295322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1534b-c185-4104-a8ba-d73f2da8abbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.296276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1534b-c185-4104-a8ba-d73f2da8abbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.296552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1534b-c185-4104-a8ba-d73f2da8abbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.296820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1534b-c185-4104-a8ba-d73f2da8abbd admin admin] PATH_INFO: `/v3/users/74d87b5ce40c42a38a59cd5194ca77aa` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.298326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a8b9d-de1a-46f8-91b7-3af88332328f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:05.298505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1534b-c185-4104-a8ba-d73f2da8abbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d87b5ce40c42a38a59cd5194ca77aa)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.300628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1534b-c185-4104-a8ba-d73f2da8abbd admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.311577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a8b9d-de1a-46f8-91b7-3af88332328f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.316548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84009e-b7e1-4b10-a59c-74d4113c80cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.317229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84009e-b7e1-4b10-a59c-74d4113c80cb admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.317512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84009e-b7e1-4b10-a59c-74d4113c80cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.317792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84009e-b7e1-4b10-a59c-74d4113c80cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.318726 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84009e-b7e1-4b10-a59c-74d4113c80cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.320594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84009e-b7e1-4b10-a59c-74d4113c80cb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.360927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 913/1806] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:05 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:05.361434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84009e-b7e1-4b10-a59c-74d4113c80cb admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:05.425536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-36b1534b-c185-4104-a8ba-d73f2da8abbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d87b5ce40c42a38a59cd5194ca77aa'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:05.448472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-36b1534b-c185-4104-a8ba-d73f2da8abbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d87b5ce40c42a38a59cd5194ca77aa'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:05.449251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-36b1534b-c185-4104-a8ba-d73f2da8abbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d87b5ce40c42a38a59cd5194ca77aa'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:05.449868 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-36b1534b-c185-4104-a8ba-d73f2da8abbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d87b5ce40c42a38a59cd5194ca77aa'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:05.460209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-36b1534b-c185-4104-a8ba-d73f2da8abbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d87b5ce40c42a38a59cd5194ca77aa'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:05.468573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84009e-b7e1-4b10-a59c-74d4113c80cb admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.482231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df774b-5a7b-489a-a67c-2ce7728d8b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.485116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 894/1807] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:05 2022] DELETE /identity/v3/users/74d87b5ce40c42a38a59cd5194ca77aa => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:05.485256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df774b-5a7b-489a-a67c-2ce7728d8b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.485256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df774b-5a7b-489a-a67c-2ce7728d8b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.485256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df774b-5a7b-489a-a67c-2ce7728d8b09 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.485610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df774b-5a7b-489a-a67c-2ce7728d8b09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.488282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df774b-5a7b-489a-a67c-2ce7728d8b09 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.491268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e56984-6bd4-40ef-9ab7-d143079d7233 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.491556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e56984-6bd4-40ef-9ab7-d143079d7233 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.491838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e56984-6bd4-40ef-9ab7-d143079d7233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.532993 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-91e56984-6bd4-40ef-9ab7-d143079d7233 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:05.532993 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-91e56984-6bd4-40ef-9ab7-d143079d7233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:05.536641 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 914/1808] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:05 2022] POST /identity/v3/users => generated 301 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:05.541968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df774b-5a7b-489a-a67c-2ce7728d8b09 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:05.547477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-05df774b-5a7b-489a-a67c-2ce7728d8b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:05.570590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df774b-5a7b-489a-a67c-2ce7728d8b09 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.618704 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 895/1809] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:05.626622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e56984-6bd4-40ef-9ab7-d143079d7233 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:05.628188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-91e56984-6bd4-40ef-9ab7-d143079d7233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:05.629486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:05.639539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:05.645783 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068b547-641f-4ddd-a1ac-b6a8f215c69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.646829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068b547-641f-4ddd-a1ac-b6a8f215c69b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.647284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068b547-641f-4ddd-a1ac-b6a8f215c69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.647804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068b547-641f-4ddd-a1ac-b6a8f215c69b admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.648721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068b547-641f-4ddd-a1ac-b6a8f215c69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.650485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068b547-641f-4ddd-a1ac-b6a8f215c69b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.667104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 915/1810] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:05.671353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068b547-641f-4ddd-a1ac-b6a8f215c69b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:05.683185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068b547-641f-4ddd-a1ac-b6a8f215c69b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.690019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31d166-acbf-4ee9-95c5-2722c44e0ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.691415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d166-acbf-4ee9-95c5-2722c44e0ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.691831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d166-acbf-4ee9-95c5-2722c44e0ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.692308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d166-acbf-4ee9-95c5-2722c44e0ca2 admin admin] PATH_INFO: `/v3/projects/9268800faa244e33b1257fe5893cf294/users/0dea8842b8e24879a538355248691cff/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.714191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e56984-6bd4-40ef-9ab7-d143079d7233 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.718244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31d166-acbf-4ee9-95c5-2722c44e0ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9268800faa244e33b1257fe5893cf294, user_id=0dea8842b8e24879a538355248691cff, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.724205 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31d166-acbf-4ee9-95c5-2722c44e0ca2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.729031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f8224-9533-4d04-934a-52e10956748f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.731322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f8224-9533-4d04-934a-52e10956748f service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.731631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f8224-9533-4d04-934a-52e10956748f service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.733498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f8224-9533-4d04-934a-52e10956748f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.754521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 916/1811] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:05 2022] PUT /identity/v3/projects/9268800faa244e33b1257fe5893cf294/users/0dea8842b8e24879a538355248691cff/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:05.759367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31d166-acbf-4ee9-95c5-2722c44e0ca2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:05.769021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f8224-9533-4d04-934a-52e10956748f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.770501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f8224-9533-4d04-934a-52e10956748f service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.799180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 896/1812] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:05.903207 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31d166-acbf-4ee9-95c5-2722c44e0ca2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.909083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818aa22-d0e3-4567-80f4-642a2fb65d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.910320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818aa22-d0e3-4567-80f4-642a2fb65d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.910320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818aa22-d0e3-4567-80f4-642a2fb65d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.910523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818aa22-d0e3-4567-80f4-642a2fb65d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.911592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818aa22-d0e3-4567-80f4-642a2fb65d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.913454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818aa22-d0e3-4567-80f4-642a2fb65d9c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.929356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 917/1813] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:05.930527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f8224-9533-4d04-934a-52e10956748f service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:05.940811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f8224-9533-4d04-934a-52e10956748f service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:05.946024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e0dc9-4c73-4c9d-b650-a07aecec2066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:05.947142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e0dc9-4c73-4c9d-b650-a07aecec2066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.947472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e0dc9-4c73-4c9d-b650-a07aecec2066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.947785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e0dc9-4c73-4c9d-b650-a07aecec2066 admin admin] PATH_INFO: `/v3/projects/9268800faa244e33b1257fe5893cf294/users/0dea8842b8e24879a538355248691cff/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:05.950491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e0dc9-4c73-4c9d-b650-a07aecec2066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9268800faa244e33b1257fe5893cf294, user_id=0dea8842b8e24879a538355248691cff, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:05.953252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e0dc9-4c73-4c9d-b650-a07aecec2066 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:05.966650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 897/1814] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:05 2022] PUT /identity/v3/projects/9268800faa244e33b1257fe5893cf294/users/0dea8842b8e24879a538355248691cff/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:05.977160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52742840-be36-4dc6-8037-677cc3ba4580 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:05.977687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52742840-be36-4dc6-8037-677cc3ba4580 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:05.978085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52742840-be36-4dc6-8037-677cc3ba4580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:06.007438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-52742840-be36-4dc6-8037-677cc3ba4580 None None] MFA Rules not processed for user `0dea8842b8e24879a538355248691cff`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:06.009113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-52742840-be36-4dc6-8037-677cc3ba4580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:06.104597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 918/1815] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:06.115487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e0dc9-4c73-4c9d-b650-a07aecec2066 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:06.117977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-926e0dc9-4c73-4c9d-b650-a07aecec2066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:06.156194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52742840-be36-4dc6-8037-677cc3ba4580 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:06.158054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-52742840-be36-4dc6-8037-677cc3ba4580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:06.196899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e0dc9-4c73-4c9d-b650-a07aecec2066 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:06.201070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf7049-9f05-4267-a720-7889e6488799 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:06.201728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf7049-9f05-4267-a720-7889e6488799 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:06.202065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf7049-9f05-4267-a720-7889e6488799 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:06.202342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf7049-9f05-4267-a720-7889e6488799 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:06.212909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:06.213078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf7049-9f05-4267-a720-7889e6488799 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:06.214673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf7049-9f05-4267-a720-7889e6488799 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:06.225982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 898/1816] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:06.234050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52742840-be36-4dc6-8037-677cc3ba4580 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:06.239250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:06.240117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:06.240397 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:06.240672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] PATH_INFO: `/v3/projects/c4ce1b344f7b4b86a42016390775f468` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:06.241850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4ce1b344f7b4b86a42016390775f468)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:06.243819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:06.287172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf7049-9f05-4267-a720-7889e6488799 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:06.289493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-aeaf7049-9f05-4267-a720-7889e6488799 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:06.295764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4ce1b344f7b4b86a42016390775f468', 'request_id': 'req-ff11b8b6-3d98-4479-99a8-888b0015a6c5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:06.302437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf7049-9f05-4267-a720-7889e6488799 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:06.305785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4ce1b344f7b4b86a42016390775f468', 'request_id': 'req-ff11b8b6-3d98-4479-99a8-888b0015a6c5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:06.313430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4ce1b344f7b4b86a42016390775f468', 'request_id': 'req-ff11b8b6-3d98-4479-99a8-888b0015a6c5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:06.328281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] The token cache is being invalidate because project c4ce1b344f7b4b86a42016390775f468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:06.328612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:06.329369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:06.331913 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 919/1817] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:06 2022] DELETE /identity/v3/projects/c4ce1b344f7b4b86a42016390775f468 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:06.336473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:06.339688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:06.373075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab183c-4f61-4c91-9af4-9701cf4d2436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:06.373850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab183c-4f61-4c91-9af4-9701cf4d2436 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:06.374823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab183c-4f61-4c91-9af4-9701cf4d2436 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:06.375221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab183c-4f61-4c91-9af4-9701cf4d2436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:06.377639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8eab183c-4f61-4c91-9af4-9701cf4d2436 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:06.421192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11b8b6-3d98-4479-99a8-888b0015a6c5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:06.428868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05634e-3a02-47f8-8419-5a707694900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:06.429778 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05634e-3a02-47f8-8419-5a707694900e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:06.430121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05634e-3a02-47f8-8419-5a707694900e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:06.430891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05634e-3a02-47f8-8419-5a707694900e admin admin] PATH_INFO: `/v3/users/f91fc14906f94aabb21b3cc9e36a7c21` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:06.432200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05634e-3a02-47f8-8419-5a707694900e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f91fc14906f94aabb21b3cc9e36a7c21)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:06.434934 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05634e-3a02-47f8-8419-5a707694900e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:06.452243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:06.455167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab183c-4f61-4c91-9af4-9701cf4d2436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:06.456774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab183c-4f61-4c91-9af4-9701cf4d2436 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:06.474103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 899/1818] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:06.486599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bf05634e-3a02-47f8-8419-5a707694900e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f91fc14906f94aabb21b3cc9e36a7c21'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:06.498318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bf05634e-3a02-47f8-8419-5a707694900e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f91fc14906f94aabb21b3cc9e36a7c21'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:06.499166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bf05634e-3a02-47f8-8419-5a707694900e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f91fc14906f94aabb21b3cc9e36a7c21'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:06.507261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bf05634e-3a02-47f8-8419-5a707694900e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f91fc14906f94aabb21b3cc9e36a7c21'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:06.507961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bf05634e-3a02-47f8-8419-5a707694900e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f91fc14906f94aabb21b3cc9e36a7c21'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:06.509962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:06.527400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 920/1819] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:06 2022] DELETE /identity/v3/users/f91fc14906f94aabb21b3cc9e36a7c21 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:07.007801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab183c-4f61-4c91-9af4-9701cf4d2436 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:07.029068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab183c-4f61-4c91-9af4-9701cf4d2436 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:07.117581 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:07.118484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.118801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.119091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] PATH_INFO: `/v3/projects/6daaf7e5026441759a0a7b2d663fa92f` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.120291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6daaf7e5026441759a0a7b2d663fa92f)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:07.122216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:07.171906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6daaf7e5026441759a0a7b2d663fa92f', 'request_id': 'req-089c903c-6f16-4dc6-acf7-cff9d9b501cb'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:07.179842 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6daaf7e5026441759a0a7b2d663fa92f', 'request_id': 'req-089c903c-6f16-4dc6-acf7-cff9d9b501cb'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:07.185930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6daaf7e5026441759a0a7b2d663fa92f', 'request_id': 'req-089c903c-6f16-4dc6-acf7-cff9d9b501cb'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:07.198125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] The token cache is being invalidate because project 6daaf7e5026441759a0a7b2d663fa92f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:07.198341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:07.199037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-089c903c-6f16-4dc6-acf7-cff9d9b501cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:07.200636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 900/1820] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:07 2022] DELETE /identity/v3/projects/6daaf7e5026441759a0a7b2d663fa92f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:07.205841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeafd25-9342-410c-b839-180b6d4b54f0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.206336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeafd25-9342-410c-b839-180b6d4b54f0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.206547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeafd25-9342-410c-b839-180b6d4b54f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.226358 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1aeafd25-9342-410c-b839-180b6d4b54f0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:07.227587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1aeafd25-9342-410c-b839-180b6d4b54f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:07.295142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 921/1821] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:07.303404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ddcaa-c8c8-4808-a85e-d44cd61c9e6d None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.303691 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ddcaa-c8c8-4808-a85e-d44cd61c9e6d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.303969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ddcaa-c8c8-4808-a85e-d44cd61c9e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.323226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-328ddcaa-c8c8-4808-a85e-d44cd61c9e6d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:07.324454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-328ddcaa-c8c8-4808-a85e-d44cd61c9e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:07.333725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 901/1822] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:07.338637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeafd25-9342-410c-b839-180b6d4b54f0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:07.349592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeafd25-9342-410c-b839-180b6d4b54f0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:07.355152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def25fa-9be8-426e-bfad-6c8554f5e80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:07.355819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def25fa-9be8-426e-bfad-6c8554f5e80e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.356099 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def25fa-9be8-426e-bfad-6c8554f5e80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.356371 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def25fa-9be8-426e-bfad-6c8554f5e80e admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.357107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def25fa-9be8-426e-bfad-6c8554f5e80e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:07.357413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def25fa-9be8-426e-bfad-6c8554f5e80e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:07.359084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def25fa-9be8-426e-bfad-6c8554f5e80e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:07.371155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 922/1823] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:07.375849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ddcaa-c8c8-4808-a85e-d44cd61c9e6d None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:07.388665 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ddcaa-c8c8-4808-a85e-d44cd61c9e6d None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:07.393661 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b6de0-5169-4bbb-a164-a31b13b751e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:07.394332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b6de0-5169-4bbb-a164-a31b13b751e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.394638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b6de0-5169-4bbb-a164-a31b13b751e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.395015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b6de0-5169-4bbb-a164-a31b13b751e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.395606 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b6de0-5169-4bbb-a164-a31b13b751e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:07.397512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b6de0-5169-4bbb-a164-a31b13b751e9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:07.429279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 902/1824] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:07 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:07.432490 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def25fa-9be8-426e-bfad-6c8554f5e80e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:07.445247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def25fa-9be8-426e-bfad-6c8554f5e80e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:07.507964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21201c52-6eff-42ff-8f8f-97823964a63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:07.508546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21201c52-6eff-42ff-8f8f-97823964a63e admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.508823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21201c52-6eff-42ff-8f8f-97823964a63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.509077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21201c52-6eff-42ff-8f8f-97823964a63e admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.509760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21201c52-6eff-42ff-8f8f-97823964a63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:07.511728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21201c52-6eff-42ff-8f8f-97823964a63e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:07.533541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 923/1825] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:07 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:07.537768 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b6de0-5169-4bbb-a164-a31b13b751e9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:07.548705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b6de0-5169-4bbb-a164-a31b13b751e9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:07.553892 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664954a4-5423-4f54-a73a-62aec78844f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:07.554585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664954a4-5423-4f54-a73a-62aec78844f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.554871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664954a4-5423-4f54-a73a-62aec78844f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.555145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664954a4-5423-4f54-a73a-62aec78844f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.555877 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664954a4-5423-4f54-a73a-62aec78844f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:07.557505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664954a4-5423-4f54-a73a-62aec78844f0 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:07.569813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 903/1826] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:07.573973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21201c52-6eff-42ff-8f8f-97823964a63e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:07.585062 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21201c52-6eff-42ff-8f8f-97823964a63e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:07.591472 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fbdb2-0e60-4667-a963-abbe4768e3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:07.592146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fbdb2-0e60-4667-a963-abbe4768e3fb admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.592424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fbdb2-0e60-4667-a963-abbe4768e3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.592678 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fbdb2-0e60-4667-a963-abbe4768e3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.593385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fbdb2-0e60-4667-a963-abbe4768e3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:07.595097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fbdb2-0e60-4667-a963-abbe4768e3fb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:07.605899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 924/1827] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:07.610177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664954a4-5423-4f54-a73a-62aec78844f0 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:07.620649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664954a4-5423-4f54-a73a-62aec78844f0 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:07.626542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d58632-d8c9-4fdb-bfa0-719e2061b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:07.627300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58632-d8c9-4fdb-bfa0-719e2061b98f admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.627578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58632-d8c9-4fdb-bfa0-719e2061b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.627848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58632-d8c9-4fdb-bfa0-719e2061b98f admin admin] PATH_INFO: `/v3/projects/796b257d8a8242fb833a3e8d52cf3339/users/61db852dada64423982e5c9724fdeb67/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.642980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d58632-d8c9-4fdb-bfa0-719e2061b98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=796b257d8a8242fb833a3e8d52cf3339, user_id=61db852dada64423982e5c9724fdeb67, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:07.645696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d58632-d8c9-4fdb-bfa0-719e2061b98f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:07.661176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 904/1828] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:07 2022] PUT /identity/v3/projects/796b257d8a8242fb833a3e8d52cf3339/users/61db852dada64423982e5c9724fdeb67/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:07.676392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41cb4-0635-40b7-8d4e-0035ea83bb1f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.676676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41cb4-0635-40b7-8d4e-0035ea83bb1f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.676953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41cb4-0635-40b7-8d4e-0035ea83bb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.706420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fbd41cb4-0635-40b7-8d4e-0035ea83bb1f None None] MFA Rules not processed for user `61db852dada64423982e5c9724fdeb67`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:07.707681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fbd41cb4-0635-40b7-8d4e-0035ea83bb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:07.758128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:07.779723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 925/1829] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:07.783852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d58632-d8c9-4fdb-bfa0-719e2061b98f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:07.813951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:07.858962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d58632-d8c9-4fdb-bfa0-719e2061b98f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:07.864505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69012f8-6d35-4b94-a245-a34542f5c84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:07.865830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69012f8-6d35-4b94-a245-a34542f5c84a admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.865830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69012f8-6d35-4b94-a245-a34542f5c84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.865830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69012f8-6d35-4b94-a245-a34542f5c84a admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.866407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69012f8-6d35-4b94-a245-a34542f5c84a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:07.868357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69012f8-6d35-4b94-a245-a34542f5c84a admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:07.903376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 905/1830] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:07 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:07.907361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41cb4-0635-40b7-8d4e-0035ea83bb1f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:07.920900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41cb4-0635-40b7-8d4e-0035ea83bb1f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:07.994954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6d0ff-0c60-4e74-aa71-25b0514fe5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:07.995752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6d0ff-0c60-4e74-aa71-25b0514fe5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:07.995910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6d0ff-0c60-4e74-aa71-25b0514fe5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:07.996180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6d0ff-0c60-4e74-aa71-25b0514fe5d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:07.996896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6d0ff-0c60-4e74-aa71-25b0514fe5d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:07.998856 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6d0ff-0c60-4e74-aa71-25b0514fe5d3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:08.033751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 926/1831] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:07 2022] POST /identity/v3/users => generated 297 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:08.035671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69012f8-6d35-4b94-a245-a34542f5c84a admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:08.047011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69012f8-6d35-4b94-a245-a34542f5c84a admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:08.053108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b6e35-c859-40b6-80c2-024f7c6f4010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:08.053985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b6e35-c859-40b6-80c2-024f7c6f4010 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:08.054403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b6e35-c859-40b6-80c2-024f7c6f4010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:08.054958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b6e35-c859-40b6-80c2-024f7c6f4010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:08.057835 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b6e35-c859-40b6-80c2-024f7c6f4010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:08.059499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b6e35-c859-40b6-80c2-024f7c6f4010 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:08.073168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 906/1832] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:08.077369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6d0ff-0c60-4e74-aa71-25b0514fe5d3 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:08.089774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6d0ff-0c60-4e74-aa71-25b0514fe5d3 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:08.095604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68ad86-14dd-4216-bf8f-241eadc8c977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:08.096433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68ad86-14dd-4216-bf8f-241eadc8c977 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:08.096785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68ad86-14dd-4216-bf8f-241eadc8c977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:08.097199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68ad86-14dd-4216-bf8f-241eadc8c977 admin admin] PATH_INFO: `/v3/projects/03aa388a34bf4f5fbb9a888cecabcba9/users/a43ffb9b2c3b49e38a27a3a528528de6/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:08.117337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68ad86-14dd-4216-bf8f-241eadc8c977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03aa388a34bf4f5fbb9a888cecabcba9, user_id=a43ffb9b2c3b49e38a27a3a528528de6, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:08.120133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68ad86-14dd-4216-bf8f-241eadc8c977 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:08.141755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 927/1833] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:08 2022] PUT /identity/v3/projects/03aa388a34bf4f5fbb9a888cecabcba9/users/a43ffb9b2c3b49e38a27a3a528528de6/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:08.145805 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b6e35-c859-40b6-80c2-024f7c6f4010 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:08.155951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b6e35-c859-40b6-80c2-024f7c6f4010 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:08.215250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0941dd-acaa-43ee-b638-70157c19ea1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:08.215960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0941dd-acaa-43ee-b638-70157c19ea1f admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:08.216351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0941dd-acaa-43ee-b638-70157c19ea1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:08.216647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0941dd-acaa-43ee-b638-70157c19ea1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:08.217465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0941dd-acaa-43ee-b638-70157c19ea1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:08.219132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0941dd-acaa-43ee-b638-70157c19ea1f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:08.231099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 907/1834] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:08.235557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68ad86-14dd-4216-bf8f-241eadc8c977 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:08.247236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68ad86-14dd-4216-bf8f-241eadc8c977 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:08.252911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ab12a-d2a8-46b1-822f-2351c526e688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:08.253828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ab12a-d2a8-46b1-822f-2351c526e688 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:08.254087 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ab12a-d2a8-46b1-822f-2351c526e688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:08.254457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ab12a-d2a8-46b1-822f-2351c526e688 admin admin] PATH_INFO: `/v3/projects/03aa388a34bf4f5fbb9a888cecabcba9/users/a43ffb9b2c3b49e38a27a3a528528de6/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:08.256511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ab12a-d2a8-46b1-822f-2351c526e688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03aa388a34bf4f5fbb9a888cecabcba9, user_id=a43ffb9b2c3b49e38a27a3a528528de6, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:08.259173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ab12a-d2a8-46b1-822f-2351c526e688 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:08.270300 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 928/1835] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:08 2022] PUT /identity/v3/projects/03aa388a34bf4f5fbb9a888cecabcba9/users/a43ffb9b2c3b49e38a27a3a528528de6/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:08.280556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4368767-e592-421e-9e25-30c780af099f None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:08.280858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4368767-e592-421e-9e25-30c780af099f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:08.281132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4368767-e592-421e-9e25-30c780af099f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:08.310231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-d4368767-e592-421e-9e25-30c780af099f None None] MFA Rules not processed for user `a43ffb9b2c3b49e38a27a3a528528de6`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:08.311545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d4368767-e592-421e-9e25-30c780af099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:08.400540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 908/1836] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:08.413548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ab12a-d2a8-46b1-822f-2351c526e688 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:08.415405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-159ab12a-d2a8-46b1-822f-2351c526e688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:08.476230 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ab12a-d2a8-46b1-822f-2351c526e688 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:08.480767 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ea28e-daa9-4b6d-8ab9-15369d2f3333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:08.481405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ea28e-daa9-4b6d-8ab9-15369d2f3333 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:08.481681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ea28e-daa9-4b6d-8ab9-15369d2f3333 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:08.481964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ea28e-daa9-4b6d-8ab9-15369d2f3333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:08.492383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ea28e-daa9-4b6d-8ab9-15369d2f3333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:08.493821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ea28e-daa9-4b6d-8ab9-15369d2f3333 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:08.504043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 929/1837] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:08.851701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4368767-e592-421e-9e25-30c780af099f None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:08.861214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4368767-e592-421e-9e25-30c780af099f None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:08.866350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4188ab4-bab4-45f0-b168-de7924079466 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a43ffb9b2c3b49e38a27a3a528528de6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03aa388a34bf4f5fbb9a888cecabcba9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:08.867062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4188ab4-bab4-45f0-b168-de7924079466 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:08.867348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4188ab4-bab4-45f0-b168-de7924079466 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:08.867623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4188ab4-bab4-45f0-b168-de7924079466 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:08.868366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4188ab4-bab4-45f0-b168-de7924079466 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:08.868676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4188ab4-bab4-45f0-b168-de7924079466 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:08.870288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4188ab4-bab4-45f0-b168-de7924079466 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:08.882460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 909/1838] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:08.886725 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ea28e-daa9-4b6d-8ab9-15369d2f3333 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:08.896687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ea28e-daa9-4b6d-8ab9-15369d2f3333 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:08.902266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb916f2d-0f5e-4861-82f9-6ba1ee673697 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a43ffb9b2c3b49e38a27a3a528528de6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03aa388a34bf4f5fbb9a888cecabcba9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:08.902963 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb916f2d-0f5e-4861-82f9-6ba1ee673697 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:08.903247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb916f2d-0f5e-4861-82f9-6ba1ee673697 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:08.903602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb916f2d-0f5e-4861-82f9-6ba1ee673697 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:08.904240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb916f2d-0f5e-4861-82f9-6ba1ee673697 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:08.906206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb916f2d-0f5e-4861-82f9-6ba1ee673697 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:08.934770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 930/1839] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:08 2022] POST /identity/v3/projects => generated 355 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:08.938956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4188ab4-bab4-45f0-b168-de7924079466 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:08.945344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:08.949058 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4188ab4-bab4-45f0-b168-de7924079466 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:09.007531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774a035-49a1-4999-beda-3f7b16ccf834 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a43ffb9b2c3b49e38a27a3a528528de6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03aa388a34bf4f5fbb9a888cecabcba9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:09.008214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774a035-49a1-4999-beda-3f7b16ccf834 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:09.008493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774a035-49a1-4999-beda-3f7b16ccf834 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:09.008769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774a035-49a1-4999-beda-3f7b16ccf834 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:09.009503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774a035-49a1-4999-beda-3f7b16ccf834 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:09.009812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774a035-49a1-4999-beda-3f7b16ccf834 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:09.011457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774a035-49a1-4999-beda-3f7b16ccf834 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:09.023357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 910/1840] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:09.081113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb916f2d-0f5e-4861-82f9-6ba1ee673697 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:09.095912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb916f2d-0f5e-4861-82f9-6ba1ee673697 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:09.102730 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a43ffb9b2c3b49e38a27a3a528528de6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03aa388a34bf4f5fbb9a888cecabcba9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:09.102993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:09.103271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:09.103585 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] PATH_INFO: `/v3/projects/9bb993da219d42b482467dda032c19c6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:09.104927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9bb993da219d42b482467dda032c19c6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:09.107131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:09.134218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:09.189119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb993da219d42b482467dda032c19c6', 'request_id': 'req-9bff32af-4a80-4829-962f-54e1144ec51c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:09.201694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb993da219d42b482467dda032c19c6', 'request_id': 'req-9bff32af-4a80-4829-962f-54e1144ec51c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:09.208698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb993da219d42b482467dda032c19c6', 'request_id': 'req-9bff32af-4a80-4829-962f-54e1144ec51c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:09.222197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] The token cache is being invalidate because project 9bb993da219d42b482467dda032c19c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:09.222480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:09.223184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9bff32af-4a80-4829-962f-54e1144ec51c tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:09.224865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 931/1841] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:09 2022] DELETE /identity/v3/projects/9bb993da219d42b482467dda032c19c6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:09.463045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774a035-49a1-4999-beda-3f7b16ccf834 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:09.465351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5774a035-49a1-4999-beda-3f7b16ccf834 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:09.549434 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774a035-49a1-4999-beda-3f7b16ccf834 tempest-VolumeQuotaClassesTest-1072635083 tempest-VolumeQuotaClassesTest-1072635083-project-admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:09.606425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:09.607267 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:09.607543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:09.607811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] PATH_INFO: `/v3/users/61db852dada64423982e5c9724fdeb67` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:09.609007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61db852dada64423982e5c9724fdeb67)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:09.611056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:09.660705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61db852dada64423982e5c9724fdeb67'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:09.661821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:09.676676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61db852dada64423982e5c9724fdeb67'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:09.677412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61db852dada64423982e5c9724fdeb67'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:09.678121 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61db852dada64423982e5c9724fdeb67'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:09.685157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61db852dada64423982e5c9724fdeb67'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:09.699161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 911/1842] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:09 2022] DELETE /identity/v3/users/61db852dada64423982e5c9724fdeb67 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:09.704018 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534040ee-3d06-4687-a578-a24cae95729a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:09.704732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534040ee-3d06-4687-a578-a24cae95729a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:09.704732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534040ee-3d06-4687-a578-a24cae95729a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:09.724187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-534040ee-3d06-4687-a578-a24cae95729a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:09.725413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-534040ee-3d06-4687-a578-a24cae95729a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:09.786863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 932/1843] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:09.797288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:09.798875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:09.865926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b64597-9304-4396-8983-8522ee102ba9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:09.870123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881acfd-9b2d-43dc-ad35-41d428f6dbd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:09.870837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881acfd-9b2d-43dc-ad35-41d428f6dbd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:09.871127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881acfd-9b2d-43dc-ad35-41d428f6dbd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:09.871347 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881acfd-9b2d-43dc-ad35-41d428f6dbd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:09.881930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881acfd-9b2d-43dc-ad35-41d428f6dbd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:09.883590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881acfd-9b2d-43dc-ad35-41d428f6dbd5 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:09.894388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 912/1844] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:10.216049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534040ee-3d06-4687-a578-a24cae95729a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:10.217531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-534040ee-3d06-4687-a578-a24cae95729a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:10.231624 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534040ee-3d06-4687-a578-a24cae95729a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:10.250620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:10.252622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:10.252902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:10.253182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] PATH_INFO: `/v3/projects/796b257d8a8242fb833a3e8d52cf3339` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:10.254444 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=796b257d8a8242fb833a3e8d52cf3339)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:10.258065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:10.347228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796b257d8a8242fb833a3e8d52cf3339', 'request_id': 'req-72a3985a-eaba-41b1-8c2a-803fd65d3936'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.369497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796b257d8a8242fb833a3e8d52cf3339', 'request_id': 'req-72a3985a-eaba-41b1-8c2a-803fd65d3936'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.397209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796b257d8a8242fb833a3e8d52cf3339', 'request_id': 'req-72a3985a-eaba-41b1-8c2a-803fd65d3936'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.427179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] The token cache is being invalidate because project 796b257d8a8242fb833a3e8d52cf3339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:10.428494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.430260 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.435259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 933/1845] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:10 2022] DELETE /identity/v3/projects/796b257d8a8242fb833a3e8d52cf3339 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:10.437443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881acfd-9b2d-43dc-ad35-41d428f6dbd5 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:10.452399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881acfd-9b2d-43dc-ad35-41d428f6dbd5 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:10.567520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df83936-dea6-4b72-bf6e-5f7b146ab8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:10.568637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df83936-dea6-4b72-bf6e-5f7b146ab8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:10.569760 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df83936-dea6-4b72-bf6e-5f7b146ab8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:10.570401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df83936-dea6-4b72-bf6e-5f7b146ab8d8 admin admin] PATH_INFO: `/v3/users/a43ffb9b2c3b49e38a27a3a528528de6` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:10.572564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df83936-dea6-4b72-bf6e-5f7b146ab8d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a43ffb9b2c3b49e38a27a3a528528de6)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:10.574631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df83936-dea6-4b72-bf6e-5f7b146ab8d8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:10.626597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:10.629305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:10.653973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3985a-eaba-41b1-8c2a-803fd65d3936 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:10.668687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:10.670143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:10.671993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:10.672580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9df83936-dea6-4b72-bf6e-5f7b146ab8d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a43ffb9b2c3b49e38a27a3a528528de6'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.672941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] PATH_INFO: `/v3/users/b541e5aecaeb427dbc36619730d8a92d` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:10.677367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b541e5aecaeb427dbc36619730d8a92d)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:10.681883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:10.687279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9df83936-dea6-4b72-bf6e-5f7b146ab8d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a43ffb9b2c3b49e38a27a3a528528de6'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.687953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9df83936-dea6-4b72-bf6e-5f7b146ab8d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a43ffb9b2c3b49e38a27a3a528528de6'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.688714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9df83936-dea6-4b72-bf6e-5f7b146ab8d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a43ffb9b2c3b49e38a27a3a528528de6'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.694456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:10.700845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9df83936-dea6-4b72-bf6e-5f7b146ab8d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a43ffb9b2c3b49e38a27a3a528528de6'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.727236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:10.741478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 913/1846] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:10 2022] DELETE /identity/v3/users/a43ffb9b2c3b49e38a27a3a528528de6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:10.776297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b541e5aecaeb427dbc36619730d8a92d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.792470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b541e5aecaeb427dbc36619730d8a92d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.793259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b541e5aecaeb427dbc36619730d8a92d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.803409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b541e5aecaeb427dbc36619730d8a92d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.804147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b541e5aecaeb427dbc36619730d8a92d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:10.838628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 934/1847] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:10 2022] DELETE /identity/v3/users/b541e5aecaeb427dbc36619730d8a92d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:10.843475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61acd-be4c-4909-a1e0-9bba41d170e7 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:10.844694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61acd-be4c-4909-a1e0-9bba41d170e7 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:10.844694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61acd-be4c-4909-a1e0-9bba41d170e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:10.869246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-e4f61acd-be4c-4909-a1e0-9bba41d170e7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:10.870484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e4f61acd-be4c-4909-a1e0-9bba41d170e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:10.949239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 914/1848] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:10.970795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:10.972445 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:11.047284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c6dfc-e176-4445-847e-449d0083bdaa admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:11.055663 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900fed4-1620-4698-9b58-3f7c603e5506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:11.056533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900fed4-1620-4698-9b58-3f7c603e5506 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:11.056877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900fed4-1620-4698-9b58-3f7c603e5506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:11.057232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900fed4-1620-4698-9b58-3f7c603e5506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:11.072769 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900fed4-1620-4698-9b58-3f7c603e5506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:11.074281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900fed4-1620-4698-9b58-3f7c603e5506 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:11.085754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 935/1849] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:11.107438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f61acd-be4c-4909-a1e0-9bba41d170e7 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:11.109916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e4f61acd-be4c-4909-a1e0-9bba41d170e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:11.133412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f61acd-be4c-4909-a1e0-9bba41d170e7 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:11.139743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:11.140626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:11.141091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:11.141310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] PATH_INFO: `/v3/projects/03aa388a34bf4f5fbb9a888cecabcba9` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:11.142926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=03aa388a34bf4f5fbb9a888cecabcba9)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:11.145430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:11.201528 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03aa388a34bf4f5fbb9a888cecabcba9', 'request_id': 'req-1a10f19e-732f-4531-9c47-5eb6f65fef4b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:11.222134 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03aa388a34bf4f5fbb9a888cecabcba9', 'request_id': 'req-1a10f19e-732f-4531-9c47-5eb6f65fef4b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:11.229536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03aa388a34bf4f5fbb9a888cecabcba9', 'request_id': 'req-1a10f19e-732f-4531-9c47-5eb6f65fef4b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:11.243642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] The token cache is being invalidate because project 03aa388a34bf4f5fbb9a888cecabcba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:11.244045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:11.244790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1a10f19e-732f-4531-9c47-5eb6f65fef4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:11.247416 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 915/1850] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:11 2022] DELETE /identity/v3/projects/03aa388a34bf4f5fbb9a888cecabcba9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:11.252601 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda48ab3-28b3-44ac-a5e6-b517b83846fe None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:11.252887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda48ab3-28b3-44ac-a5e6-b517b83846fe None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:11.253172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda48ab3-28b3-44ac-a5e6-b517b83846fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:11.274683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dda48ab3-28b3-44ac-a5e6-b517b83846fe None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:11.276114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dda48ab3-28b3-44ac-a5e6-b517b83846fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:11.363584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 936/1851] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:11.370048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4888a9-9af8-40a0-bbab-6f44fbdac440 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:11.370341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4888a9-9af8-40a0-bbab-6f44fbdac440 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:11.370652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4888a9-9af8-40a0-bbab-6f44fbdac440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:11.403444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-bc4888a9-9af8-40a0-bbab-6f44fbdac440 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:11.404807 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-bc4888a9-9af8-40a0-bbab-6f44fbdac440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:11.420946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 916/1852] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:11.425340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda48ab3-28b3-44ac-a5e6-b517b83846fe None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:11.445286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda48ab3-28b3-44ac-a5e6-b517b83846fe None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:11.459252 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad39192-e7ac-4aaf-88be-6658e9442617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:11.460241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad39192-e7ac-4aaf-88be-6658e9442617 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:11.461286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad39192-e7ac-4aaf-88be-6658e9442617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:11.463798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad39192-e7ac-4aaf-88be-6658e9442617 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:11.464764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad39192-e7ac-4aaf-88be-6658e9442617 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:11.465304 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad39192-e7ac-4aaf-88be-6658e9442617 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:11.469344 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad39192-e7ac-4aaf-88be-6658e9442617 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:11.509248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 937/1853] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:11.511473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4888a9-9af8-40a0-bbab-6f44fbdac440 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:11.532928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4888a9-9af8-40a0-bbab-6f44fbdac440 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:11.546814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fcf79-2a66-480d-ad5f-27616ba9059c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:11.547649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fcf79-2a66-480d-ad5f-27616ba9059c admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:11.547931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fcf79-2a66-480d-ad5f-27616ba9059c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:11.549052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fcf79-2a66-480d-ad5f-27616ba9059c admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:11.549052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fcf79-2a66-480d-ad5f-27616ba9059c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:11.553167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fcf79-2a66-480d-ad5f-27616ba9059c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:11.619503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 917/1854] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:11 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:11.620565 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad39192-e7ac-4aaf-88be-6658e9442617 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:11.632221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad39192-e7ac-4aaf-88be-6658e9442617 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:11.680450 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:11.736871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bd432-df62-4040-9043-b12790909051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:11.737624 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bd432-df62-4040-9043-b12790909051 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:11.737854 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bd432-df62-4040-9043-b12790909051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:11.738103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bd432-df62-4040-9043-b12790909051 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:11.738903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bd432-df62-4040-9043-b12790909051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:11.740927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bd432-df62-4040-9043-b12790909051 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:11.794467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 938/1855] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:11 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:11.796200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fcf79-2a66-480d-ad5f-27616ba9059c admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:11.807439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fcf79-2a66-480d-ad5f-27616ba9059c admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:11.820054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a623e-54d4-4064-bb74-46b41e56e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:11.820756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a623e-54d4-4064-bb74-46b41e56e357 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:11.821751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a623e-54d4-4064-bb74-46b41e56e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:11.824309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a623e-54d4-4064-bb74-46b41e56e357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:11.825232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a623e-54d4-4064-bb74-46b41e56e357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:11.830427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a623e-54d4-4064-bb74-46b41e56e357 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:11.861788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bd432-df62-4040-9043-b12790909051 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:11.867472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 918/1856] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:11.873066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bd432-df62-4040-9043-b12790909051 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:11.879041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd14299-c366-42df-becc-0ed993a18f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:11.879824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd14299-c366-42df-becc-0ed993a18f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:11.880176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd14299-c366-42df-becc-0ed993a18f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:11.880518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd14299-c366-42df-becc-0ed993a18f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:11.881325 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd14299-c366-42df-becc-0ed993a18f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:11.883323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd14299-c366-42df-becc-0ed993a18f68 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:11.900296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 939/1857] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:11.904785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a623e-54d4-4064-bb74-46b41e56e357 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:11.921377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a623e-54d4-4064-bb74-46b41e56e357 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:11.934239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16966547-3d55-4348-8eee-724df2abc14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:11.935221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16966547-3d55-4348-8eee-724df2abc14b admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:11.935484 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16966547-3d55-4348-8eee-724df2abc14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:11.935757 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16966547-3d55-4348-8eee-724df2abc14b admin admin] PATH_INFO: `/v3/projects/14b8267cc8e44486b600acc0f481076b/users/c5294304f59e4635ab922885295c891a/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:11.968641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16966547-3d55-4348-8eee-724df2abc14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b8267cc8e44486b600acc0f481076b, user_id=c5294304f59e4635ab922885295c891a, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:11.971485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16966547-3d55-4348-8eee-724df2abc14b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:12.005129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 919/1858] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:11 2022] PUT /identity/v3/projects/14b8267cc8e44486b600acc0f481076b/users/c5294304f59e4635ab922885295c891a/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:12.007720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e289d-0563-456a-8eae-ed3eeae735da None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:12.008007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e289d-0563-456a-8eae-ed3eeae735da None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:12.008282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e289d-0563-456a-8eae-ed3eeae735da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:12.082408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-587e289d-0563-456a-8eae-ed3eeae735da None None] MFA Rules not processed for user `c5294304f59e4635ab922885295c891a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:12.082408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-587e289d-0563-456a-8eae-ed3eeae735da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:12.195987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16966547-3d55-4348-8eee-724df2abc14b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:12.197967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-16966547-3d55-4348-8eee-724df2abc14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:12.207432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 940/1859] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:12.214075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:12.217636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e289d-0563-456a-8eae-ed3eeae735da None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:12.298065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16966547-3d55-4348-8eee-724df2abc14b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:12.306014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:12.307249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:12.307538 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:12.308320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] PATH_INFO: `/v3/projects/7452607201cc4220aed481577666a5b2` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:12.310637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7452607201cc4220aed481577666a5b2)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:12.312594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:12.375299 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e289d-0563-456a-8eae-ed3eeae735da None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:12.383590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f894a-2e03-4af3-aef6-03e98289e673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:12.384599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f894a-2e03-4af3-aef6-03e98289e673 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:12.384709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f894a-2e03-4af3-aef6-03e98289e673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:12.385505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f894a-2e03-4af3-aef6-03e98289e673 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:12.385813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f894a-2e03-4af3-aef6-03e98289e673 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:12.388437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f894a-2e03-4af3-aef6-03e98289e673 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:12.404762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7452607201cc4220aed481577666a5b2', 'request_id': 'req-95568468-5c38-4259-8ac0-54bda4dc19a8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:12.414111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7452607201cc4220aed481577666a5b2', 'request_id': 'req-95568468-5c38-4259-8ac0-54bda4dc19a8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:12.424084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7452607201cc4220aed481577666a5b2', 'request_id': 'req-95568468-5c38-4259-8ac0-54bda4dc19a8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:12.432709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 941/1860] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:12 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:12.436514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f894a-2e03-4af3-aef6-03e98289e673 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:12.453507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] The token cache is being invalidate because project 7452607201cc4220aed481577666a5b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:12.453881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:12.455079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:12.457441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 920/1861] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:12 2022] DELETE /identity/v3/projects/7452607201cc4220aed481577666a5b2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:12.461160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:12.463965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:12.478124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:12.523845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f894a-2e03-4af3-aef6-03e98289e673 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:12.529282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51f643-a0a9-45e6-aa5b-6bc1898a09d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:12.530000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51f643-a0a9-45e6-aa5b-6bc1898a09d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:12.530325 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51f643-a0a9-45e6-aa5b-6bc1898a09d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:12.530543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51f643-a0a9-45e6-aa5b-6bc1898a09d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:12.531693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51f643-a0a9-45e6-aa5b-6bc1898a09d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:12.535433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51f643-a0a9-45e6-aa5b-6bc1898a09d8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:12.556510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95568468-5c38-4259-8ac0-54bda4dc19a8 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:12.569268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:12.570341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:12.571864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:12.573761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] PATH_INFO: `/v3/users/70d8e5e2ab574b8b89f7f94cc32e6f9a` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:12.576751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d8e5e2ab574b8b89f7f94cc32e6f9a)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:12.581872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:12.588021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 942/1862] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:12 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:12.588961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51f643-a0a9-45e6-aa5b-6bc1898a09d8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:12.592855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9a51f643-a0a9-45e6-aa5b-6bc1898a09d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:12.613661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51f643-a0a9-45e6-aa5b-6bc1898a09d8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:12.619889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900013e-3ada-4334-8235-f1b6a6fc290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:12.620591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900013e-3ada-4334-8235-f1b6a6fc290c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:12.620857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900013e-3ada-4334-8235-f1b6a6fc290c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:12.621129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900013e-3ada-4334-8235-f1b6a6fc290c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:12.622049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900013e-3ada-4334-8235-f1b6a6fc290c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:12.624335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900013e-3ada-4334-8235-f1b6a6fc290c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:12.637723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 943/1863] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:12.642069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900013e-3ada-4334-8235-f1b6a6fc290c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:12.666835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900013e-3ada-4334-8235-f1b6a6fc290c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:12.667937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:12.675578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d8e5e2ab574b8b89f7f94cc32e6f9a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:12.675771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184915f-6f39-4679-8f88-0dcd51d451e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:12.676678 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184915f-6f39-4679-8f88-0dcd51d451e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:12.677003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184915f-6f39-4679-8f88-0dcd51d451e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:12.681860 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184915f-6f39-4679-8f88-0dcd51d451e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:12.682961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184915f-6f39-4679-8f88-0dcd51d451e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:12.689008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184915f-6f39-4679-8f88-0dcd51d451e0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:12.698039 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d8e5e2ab574b8b89f7f94cc32e6f9a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:12.699519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d8e5e2ab574b8b89f7f94cc32e6f9a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:12.700781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d8e5e2ab574b8b89f7f94cc32e6f9a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:12.710832 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d8e5e2ab574b8b89f7f94cc32e6f9a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:12.730721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 944/1864] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:12.734557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184915f-6f39-4679-8f88-0dcd51d451e0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:12.740044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4184915f-6f39-4679-8f88-0dcd51d451e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:12.742425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 921/1865] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:12 2022] DELETE /identity/v3/users/70d8e5e2ab574b8b89f7f94cc32e6f9a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:12.772135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184915f-6f39-4679-8f88-0dcd51d451e0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:12.825829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:12.828553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:12.858990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22de58d-4ed6-4d4a-a386-788cfa65c495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:12.860037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22de58d-4ed6-4d4a-a386-788cfa65c495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:12.860418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22de58d-4ed6-4d4a-a386-788cfa65c495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:12.860980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22de58d-4ed6-4d4a-a386-788cfa65c495 admin admin] PATH_INFO: `/v3/projects/498e6cad85d745c990012baf92f5176e/users/c44522bbfc7f418488a11771d8b7e38c/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:12.892561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22de58d-4ed6-4d4a-a386-788cfa65c495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=498e6cad85d745c990012baf92f5176e, user_id=c44522bbfc7f418488a11771d8b7e38c, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:12.898448 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22de58d-4ed6-4d4a-a386-788cfa65c495 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:12.915580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8872c2-331a-46ea-99a8-d937ec33de29 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:12.926129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eb874-058e-4229-862d-38996b54e29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:12.928223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eb874-058e-4229-862d-38996b54e29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:12.928623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 945/1866] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:12 2022] PUT /identity/v3/projects/498e6cad85d745c990012baf92f5176e/users/c44522bbfc7f418488a11771d8b7e38c/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:12.937443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eb874-058e-4229-862d-38996b54e29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:12.937443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eb874-058e-4229-862d-38996b54e29f admin admin] PATH_INFO: `/v3/users/c9509e42085b4fa5a5ba6a2ef28e7efd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:12.939007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000eb874-058e-4229-862d-38996b54e29f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9509e42085b4fa5a5ba6a2ef28e7efd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:12.939727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d302f-0751-4546-87f2-8290066ecf51 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:12.939954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d302f-0751-4546-87f2-8290066ecf51 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:12.940242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d302f-0751-4546-87f2-8290066ecf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:12.942204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000eb874-058e-4229-862d-38996b54e29f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:12.997946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-703d302f-0751-4546-87f2-8290066ecf51 None None] MFA Rules not processed for user `c44522bbfc7f418488a11771d8b7e38c`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:12.999131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-703d302f-0751-4546-87f2-8290066ecf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:13.020283 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-000eb874-058e-4229-862d-38996b54e29f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9509e42085b4fa5a5ba6a2ef28e7efd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:13.038967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-000eb874-058e-4229-862d-38996b54e29f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9509e42085b4fa5a5ba6a2ef28e7efd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:13.039874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-000eb874-058e-4229-862d-38996b54e29f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9509e42085b4fa5a5ba6a2ef28e7efd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:13.040688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-000eb874-058e-4229-862d-38996b54e29f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9509e42085b4fa5a5ba6a2ef28e7efd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:13.051531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-000eb874-058e-4229-862d-38996b54e29f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9509e42085b4fa5a5ba6a2ef28e7efd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:13.072593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 922/1867] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:12 2022] DELETE /identity/v3/users/c9509e42085b4fa5a5ba6a2ef28e7efd => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:13.077407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd2a83-fb42-4adb-ab4e-a4c8c5db697c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:13.077816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd2a83-fb42-4adb-ab4e-a4c8c5db697c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:13.077980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd2a83-fb42-4adb-ab4e-a4c8c5db697c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:13.100823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-8bdd2a83-fb42-4adb-ab4e-a4c8c5db697c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:13.101488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8bdd2a83-fb42-4adb-ab4e-a4c8c5db697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:13.141549 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:13.175176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 946/1868] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:13.179914 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d302f-0751-4546-87f2-8290066ecf51 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:13.182471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-703d302f-0751-4546-87f2-8290066ecf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:13.188847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:13.223539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 923/1869] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:13.241783 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd2a83-fb42-4adb-ab4e-a4c8c5db697c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:13.243293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8bdd2a83-fb42-4adb-ab4e-a4c8c5db697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:13.293899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d302f-0751-4546-87f2-8290066ecf51 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:13.300802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de71ebf-394f-4e67-b839-c4a12796f5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:13.301580 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de71ebf-394f-4e67-b839-c4a12796f5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:13.301982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de71ebf-394f-4e67-b839-c4a12796f5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:13.302325 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de71ebf-394f-4e67-b839-c4a12796f5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:13.303326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de71ebf-394f-4e67-b839-c4a12796f5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:13.305903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de71ebf-394f-4e67-b839-c4a12796f5e0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:13.366031 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 947/1870] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:13 2022] POST /identity/v3/projects => generated 387 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:13.372060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de71ebf-394f-4e67-b839-c4a12796f5e0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:13.383067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd2a83-fb42-4adb-ab4e-a4c8c5db697c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:13.480604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f02bbe-fcab-4287-bb51-3fdfaa2b394e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:13.481290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f02bbe-fcab-4287-bb51-3fdfaa2b394e service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:13.481851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f02bbe-fcab-4287-bb51-3fdfaa2b394e service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:13.482223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f02bbe-fcab-4287-bb51-3fdfaa2b394e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:13.531897 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de71ebf-394f-4e67-b839-c4a12796f5e0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:13.545816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc83a8-6895-4fed-a2cf-cceeaf98f070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:13.546940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc83a8-6895-4fed-a2cf-cceeaf98f070 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:13.547228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc83a8-6895-4fed-a2cf-cceeaf98f070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:13.547504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc83a8-6895-4fed-a2cf-cceeaf98f070 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:13.548329 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc83a8-6895-4fed-a2cf-cceeaf98f070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:13.551673 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc83a8-6895-4fed-a2cf-cceeaf98f070 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:13.585412 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 948/1871] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:13 2022] POST /identity/v3/users => generated 300 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:13.595796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc83a8-6895-4fed-a2cf-cceeaf98f070 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:13.607018 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc83a8-6895-4fed-a2cf-cceeaf98f070 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:13.615212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23652246-9c57-4f21-be57-8aa7097b7bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:13.615901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23652246-9c57-4f21-be57-8aa7097b7bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:13.616182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23652246-9c57-4f21-be57-8aa7097b7bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:13.616454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23652246-9c57-4f21-be57-8aa7097b7bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:13.617198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23652246-9c57-4f21-be57-8aa7097b7bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:13.618842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23652246-9c57-4f21-be57-8aa7097b7bf3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:13.630417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 949/1872] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:13.634144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23652246-9c57-4f21-be57-8aa7097b7bf3 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:13.636161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f02bbe-fcab-4287-bb51-3fdfaa2b394e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:13.637986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f02bbe-fcab-4287-bb51-3fdfaa2b394e service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:13.645093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23652246-9c57-4f21-be57-8aa7097b7bf3 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:13.651915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f84f52-d37f-4074-b6d2-91aca3a6ecc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:13.653872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f84f52-d37f-4074-b6d2-91aca3a6ecc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:13.654233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f84f52-d37f-4074-b6d2-91aca3a6ecc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:13.654605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f84f52-d37f-4074-b6d2-91aca3a6ecc7 admin admin] PATH_INFO: `/v3/projects/1a55bc0976a4443f89eefb8350147124/users/fc4ae06588f040cfa88df9ef4d4684f3/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:13.655656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 924/1873] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:13.672458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f84f52-d37f-4074-b6d2-91aca3a6ecc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a55bc0976a4443f89eefb8350147124, user_id=fc4ae06588f040cfa88df9ef4d4684f3, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:13.675492 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f84f52-d37f-4074-b6d2-91aca3a6ecc7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:13.695712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 950/1874] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:13 2022] PUT /identity/v3/projects/1a55bc0976a4443f89eefb8350147124/users/fc4ae06588f040cfa88df9ef4d4684f3/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:13.700162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f02bbe-fcab-4287-bb51-3fdfaa2b394e service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:13.714781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f02bbe-fcab-4287-bb51-3fdfaa2b394e service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:13.768360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:13.790773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f84f52-d37f-4074-b6d2-91aca3a6ecc7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:13.792911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d8f40-84fe-4825-b7cf-8036400901c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:13.793486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b0f84f52-d37f-4074-b6d2-91aca3a6ecc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:13.794309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d8f40-84fe-4825-b7cf-8036400901c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:13.795003 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d8f40-84fe-4825-b7cf-8036400901c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:13.795841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d8f40-84fe-4825-b7cf-8036400901c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:13.796963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d8f40-84fe-4825-b7cf-8036400901c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:13.800077 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d8f40-84fe-4825-b7cf-8036400901c7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:13.819753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 925/1875] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:13.824075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d8f40-84fe-4825-b7cf-8036400901c7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:13.825089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f84f52-d37f-4074-b6d2-91aca3a6ecc7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:13.838276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:13.839310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:13.839588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:13.839865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] PATH_INFO: `/v3/projects/cf60e0942861430692b0dda6cdb773d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:13.841923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf60e0942861430692b0dda6cdb773d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:13.843982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:13.849891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d8f40-84fe-4825-b7cf-8036400901c7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:13.859814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133e5c4-c15d-44d2-9109-5dad8637b880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:13.861322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133e5c4-c15d-44d2-9109-5dad8637b880 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:13.861859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133e5c4-c15d-44d2-9109-5dad8637b880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:13.862367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133e5c4-c15d-44d2-9109-5dad8637b880 admin admin] PATH_INFO: `/v3/projects/1a55bc0976a4443f89eefb8350147124/users/fc4ae06588f040cfa88df9ef4d4684f3/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:13.866630 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133e5c4-c15d-44d2-9109-5dad8637b880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a55bc0976a4443f89eefb8350147124, user_id=fc4ae06588f040cfa88df9ef4d4684f3, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:13.870839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133e5c4-c15d-44d2-9109-5dad8637b880 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:13.891907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 926/1876] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:13 2022] PUT /identity/v3/projects/1a55bc0976a4443f89eefb8350147124/users/fc4ae06588f040cfa88df9ef4d4684f3/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:13.906556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:13.921525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf60e0942861430692b0dda6cdb773d6', 'request_id': 'req-e0a6616f-4340-4346-bb32-e941afda21ab'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:13.923034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1a6a6-e2bc-4ead-be86-3233e3b2a553 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:13.927129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1a6a6-e2bc-4ead-be86-3233e3b2a553 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:13.931932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1a6a6-e2bc-4ead-be86-3233e3b2a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:13.946297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf60e0942861430692b0dda6cdb773d6', 'request_id': 'req-e0a6616f-4340-4346-bb32-e941afda21ab'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:13.957996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf60e0942861430692b0dda6cdb773d6', 'request_id': 'req-e0a6616f-4340-4346-bb32-e941afda21ab'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:13.984875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] The token cache is being invalidate because project cf60e0942861430692b0dda6cdb773d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:13.985328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:13.986454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e0a6616f-4340-4346-bb32-e941afda21ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:13.988943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 951/1877] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:13 2022] DELETE /identity/v3/projects/cf60e0942861430692b0dda6cdb773d6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:13.992237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-3ac1a6a6-e2bc-4ead-be86-3233e3b2a553 None None] MFA Rules not processed for user `fc4ae06588f040cfa88df9ef4d4684f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:13.993796 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3ac1a6a6-e2bc-4ead-be86-3233e3b2a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:14.010190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f926ed-ed09-4f9f-bea0-1d337098d869 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:14.011203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f926ed-ed09-4f9f-bea0-1d337098d869 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:14.011588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f926ed-ed09-4f9f-bea0-1d337098d869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:14.044302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-99f926ed-ed09-4f9f-bea0-1d337098d869 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:14.046890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-99f926ed-ed09-4f9f-bea0-1d337098d869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:14.171711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 952/1878] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:14.181246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cb9a3-83e7-4851-bfed-14b9c459b2e9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:14.181536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cb9a3-83e7-4851-bfed-14b9c459b2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:14.181816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cb9a3-83e7-4851-bfed-14b9c459b2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:14.186740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 927/1879] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:14.210755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1a6a6-e2bc-4ead-be86-3233e3b2a553 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:14.212900 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3ac1a6a6-e2bc-4ead-be86-3233e3b2a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:14.240373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-381cb9a3-83e7-4851-bfed-14b9c459b2e9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:14.244603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-381cb9a3-83e7-4851-bfed-14b9c459b2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:14.268942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 953/1880] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:14.273258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cb9a3-83e7-4851-bfed-14b9c459b2e9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:14.285550 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cb9a3-83e7-4851-bfed-14b9c459b2e9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:14.291642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72577319-7db8-44e4-a7ec-46e6f2b22168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:14.292504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577319-7db8-44e4-a7ec-46e6f2b22168 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:14.292927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577319-7db8-44e4-a7ec-46e6f2b22168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:14.293361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577319-7db8-44e4-a7ec-46e6f2b22168 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:14.294254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72577319-7db8-44e4-a7ec-46e6f2b22168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:14.294768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72577319-7db8-44e4-a7ec-46e6f2b22168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:14.296653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72577319-7db8-44e4-a7ec-46e6f2b22168 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:14.316091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 954/1881] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:14.317528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72577319-7db8-44e4-a7ec-46e6f2b22168 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:14.328808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72577319-7db8-44e4-a7ec-46e6f2b22168 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:14.329759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1a6a6-e2bc-4ead-be86-3233e3b2a553 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:14.337287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97fd9c-1f69-4d6b-8b00-385be8fafa9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:14.337936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97fd9c-1f69-4d6b-8b00-385be8fafa9a service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:14.338221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97fd9c-1f69-4d6b-8b00-385be8fafa9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:14.338495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97fd9c-1f69-4d6b-8b00-385be8fafa9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:14.338774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296cbd9e-735a-441e-b132-1b8620f54da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:14.339620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cbd9e-735a-441e-b132-1b8620f54da7 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:14.340043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cbd9e-735a-441e-b132-1b8620f54da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:14.340449 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cbd9e-735a-441e-b132-1b8620f54da7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:14.341415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296cbd9e-735a-441e-b132-1b8620f54da7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:14.343547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296cbd9e-735a-441e-b132-1b8620f54da7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:14.348937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97fd9c-1f69-4d6b-8b00-385be8fafa9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:14.350474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97fd9c-1f69-4d6b-8b00-385be8fafa9a service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:14.361733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 928/1882] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:14.398192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 955/1883] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:14 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:14.401396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97fd9c-1f69-4d6b-8b00-385be8fafa9a service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:14.420351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97fd9c-1f69-4d6b-8b00-385be8fafa9a service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:14.438139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296cbd9e-735a-441e-b132-1b8620f54da7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:14.440455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-296cbd9e-735a-441e-b132-1b8620f54da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:14.557112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:14.567596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296cbd9e-735a-441e-b132-1b8620f54da7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:14.575523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5de34-df59-42b7-987f-68a0d518f2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:14.576534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5de34-df59-42b7-987f-68a0d518f2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:14.577389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5de34-df59-42b7-987f-68a0d518f2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:14.577678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5de34-df59-42b7-987f-68a0d518f2a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:14.578582 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5de34-df59-42b7-987f-68a0d518f2a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:14.579598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:14.581353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5de34-df59-42b7-987f-68a0d518f2a4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:14.582240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:14.582757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:14.583227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] PATH_INFO: `/v3/projects/506065ed60c141e7ab316b5892b41b77` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:14.584709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] RBAC: Authorizing `identity:delete_project(project_id=506065ed60c141e7ab316b5892b41b77)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:14.588168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:14.624002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 929/1884] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:14 2022] POST /identity/v3/users => generated 292 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:14.626799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5de34-df59-42b7-987f-68a0d518f2a4 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:14.647104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:14.650106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5de34-df59-42b7-987f-68a0d518f2a4 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:14.664156 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28eac8-035c-4b5a-ad54-e94a5e65e1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:14.665067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28eac8-035c-4b5a-ad54-e94a5e65e1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:14.665858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28eac8-035c-4b5a-ad54-e94a5e65e1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:14.666144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28eac8-035c-4b5a-ad54-e94a5e65e1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:14.670228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28eac8-035c-4b5a-ad54-e94a5e65e1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:14.673602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28eac8-035c-4b5a-ad54-e94a5e65e1f5 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:14.706439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 930/1885] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:14.707299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28eac8-035c-4b5a-ad54-e94a5e65e1f5 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:14.727704 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28eac8-035c-4b5a-ad54-e94a5e65e1f5 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:14.748049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506065ed60c141e7ab316b5892b41b77', 'request_id': 'req-f0757ebb-30df-4c30-a27a-962f23b6816d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:14.769135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506065ed60c141e7ab316b5892b41b77', 'request_id': 'req-f0757ebb-30df-4c30-a27a-962f23b6816d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:14.777813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506065ed60c141e7ab316b5892b41b77', 'request_id': 'req-f0757ebb-30df-4c30-a27a-962f23b6816d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:14.821594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] The token cache is being invalidate because project 506065ed60c141e7ab316b5892b41b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:14.823411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:14.826877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:14.832170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 956/1886] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:14 2022] DELETE /identity/v3/projects/506065ed60c141e7ab316b5892b41b77 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:14.837526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:14.840946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:14.912751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf75d2-fb3c-46d6-a9df-622b349c9c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:14.915310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf75d2-fb3c-46d6-a9df-622b349c9c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:14.917206 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf75d2-fb3c-46d6-a9df-622b349c9c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:14.917786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf75d2-fb3c-46d6-a9df-622b349c9c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:14.919329 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf75d2-fb3c-46d6-a9df-622b349c9c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:14.923464 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf75d2-fb3c-46d6-a9df-622b349c9c60 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:14.950598 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 931/1887] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:14.955384 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf75d2-fb3c-46d6-a9df-622b349c9c60 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:14.963658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-06bf75d2-fb3c-46d6-a9df-622b349c9c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:15.004148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf75d2-fb3c-46d6-a9df-622b349c9c60 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:15.013308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0757ebb-30df-4c30-a27a-962f23b6816d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:15.016625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a896c4-7f70-45e3-8bf0-e4d8811f2232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:15.019774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a896c4-7f70-45e3-8bf0-e4d8811f2232 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:15.020064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a896c4-7f70-45e3-8bf0-e4d8811f2232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:15.021084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a896c4-7f70-45e3-8bf0-e4d8811f2232 admin admin] PATH_INFO: `/v3/projects/913e817d36bc43f9aebaa92a8e7aad8c/users/8fee35dd25b04cd8853a14e5ff8b44ab/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:15.027557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfd717-1b70-4a1a-b91b-9d9984c493d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:15.029095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfd717-1b70-4a1a-b91b-9d9984c493d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:15.029972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfd717-1b70-4a1a-b91b-9d9984c493d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:15.031534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfd717-1b70-4a1a-b91b-9d9984c493d8 admin admin] PATH_INFO: `/v3/users/0dea8842b8e24879a538355248691cff` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:15.034225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfd717-1b70-4a1a-b91b-9d9984c493d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dea8842b8e24879a538355248691cff)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:15.038495 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfd717-1b70-4a1a-b91b-9d9984c493d8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:15.057934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a896c4-7f70-45e3-8bf0-e4d8811f2232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913e817d36bc43f9aebaa92a8e7aad8c, user_id=8fee35dd25b04cd8853a14e5ff8b44ab, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:15.063741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a896c4-7f70-45e3-8bf0-e4d8811f2232 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:15.105866 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 932/1888] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:14 2022] PUT /identity/v3/projects/913e817d36bc43f9aebaa92a8e7aad8c/users/8fee35dd25b04cd8853a14e5ff8b44ab/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:15.108406 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b12ca-3e07-4914-be2d-fb8a19598655 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:15.108772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b12ca-3e07-4914-be2d-fb8a19598655 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:15.109042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b12ca-3e07-4914-be2d-fb8a19598655 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:15.112728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 933/1889] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 09:59:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:59:15.114194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ec22f-b208-4e14-a5e3-fb7b28b1487b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:15.114483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ec22f-b208-4e14-a5e3-fb7b28b1487b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:15.114781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ec22f-b208-4e14-a5e3-fb7b28b1487b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:15.158518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5ccfd717-1b70-4a1a-b91b-9d9984c493d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dea8842b8e24879a538355248691cff'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:15.200112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5ccfd717-1b70-4a1a-b91b-9d9984c493d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dea8842b8e24879a538355248691cff'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:15.201477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5ccfd717-1b70-4a1a-b91b-9d9984c493d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dea8842b8e24879a538355248691cff'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:15.203763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-aa0ec22f-b208-4e14-a5e3-fb7b28b1487b None None] MFA Rules not processed for user `8fee35dd25b04cd8853a14e5ff8b44ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:15.207442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-aa0ec22f-b208-4e14-a5e3-fb7b28b1487b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:15.222978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5ccfd717-1b70-4a1a-b91b-9d9984c493d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dea8842b8e24879a538355248691cff'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:15.224733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5ccfd717-1b70-4a1a-b91b-9d9984c493d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dea8842b8e24879a538355248691cff'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:15.268073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 957/1890] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:14 2022] DELETE /identity/v3/users/0dea8842b8e24879a538355248691cff => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:15.269748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdae83-fae3-4800-bfe0-b8200abc6dc7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:15.270108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdae83-fae3-4800-bfe0-b8200abc6dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:15.270482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdae83-fae3-4800-bfe0-b8200abc6dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:15.329087 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-17bdae83-fae3-4800-bfe0-b8200abc6dc7 None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:15.330648 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-17bdae83-fae3-4800-bfe0-b8200abc6dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:15.403277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:15.428842 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 934/1891] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:15.435087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ec22f-b208-4e14-a5e3-fb7b28b1487b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:15.450512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-aa0ec22f-b208-4e14-a5e3-fb7b28b1487b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:15.499517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 958/1892] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 09:59:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:15.535261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdae83-fae3-4800-bfe0-b8200abc6dc7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:15.539765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-17bdae83-fae3-4800-bfe0-b8200abc6dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:15.545825 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:15.565742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdae83-fae3-4800-bfe0-b8200abc6dc7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:15.574621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaccded-6aed-4887-ad0f-bbadf5ad3c66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:15.575441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaccded-6aed-4887-ad0f-bbadf5ad3c66 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:15.575441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaccded-6aed-4887-ad0f-bbadf5ad3c66 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:15.575687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaccded-6aed-4887-ad0f-bbadf5ad3c66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:15.588973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaccded-6aed-4887-ad0f-bbadf5ad3c66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:15.590370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaccded-6aed-4887-ad0f-bbadf5ad3c66 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:15.607360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 959/1893] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:15.607926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaccded-6aed-4887-ad0f-bbadf5ad3c66 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:15.608635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ec22f-b208-4e14-a5e3-fb7b28b1487b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:15.609584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4eaccded-6aed-4887-ad0f-bbadf5ad3c66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:15.698903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707730-dca4-4e80-bb63-95e376362f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:15.700187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707730-dca4-4e80-bb63-95e376362f21 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:15.700446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707730-dca4-4e80-bb63-95e376362f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:15.700866 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707730-dca4-4e80-bb63-95e376362f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:15.701646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73707730-dca4-4e80-bb63-95e376362f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:15.704162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73707730-dca4-4e80-bb63-95e376362f21 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:15.739487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaccded-6aed-4887-ad0f-bbadf5ad3c66 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:15.744437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbed369-6a4a-4816-94e8-6b617119d95d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:15.745188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbed369-6a4a-4816-94e8-6b617119d95d service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:15.745540 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbed369-6a4a-4816-94e8-6b617119d95d service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:15.745871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbed369-6a4a-4816-94e8-6b617119d95d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:15.748615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1cbed369-6a4a-4816-94e8-6b617119d95d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:15.751490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 935/1894] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:15 2022] POST /identity/v3/projects => generated 379 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:15.754637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707730-dca4-4e80-bb63-95e376362f21 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:15.847483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbed369-6a4a-4816-94e8-6b617119d95d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:15.849291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbed369-6a4a-4816-94e8-6b617119d95d service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:15.861463 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 960/1895] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:15.900694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707730-dca4-4e80-bb63-95e376362f21 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:15.918490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1de576-1a7a-49f7-a3e2-534a6b91dc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:15.920452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1de576-1a7a-49f7-a3e2-534a6b91dc7e admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:15.921190 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1de576-1a7a-49f7-a3e2-534a6b91dc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:15.921487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1de576-1a7a-49f7-a3e2-534a6b91dc7e admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:15.922201 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1de576-1a7a-49f7-a3e2-534a6b91dc7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:15.928405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1de576-1a7a-49f7-a3e2-534a6b91dc7e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:15.960197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 936/1896] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:15 2022] POST /identity/v3/users => generated 296 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:15.964850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbed369-6a4a-4816-94e8-6b617119d95d service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:15.980415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbed369-6a4a-4816-94e8-6b617119d95d service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:15.990491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20024bed-88cd-43a3-8fc7-66f68c89172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:15.991516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20024bed-88cd-43a3-8fc7-66f68c89172d admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:15.999784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20024bed-88cd-43a3-8fc7-66f68c89172d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:16.000290 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20024bed-88cd-43a3-8fc7-66f68c89172d admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:16.001371 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20024bed-88cd-43a3-8fc7-66f68c89172d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:16.004978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20024bed-88cd-43a3-8fc7-66f68c89172d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:16.023933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 961/1897] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:16.026614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1de576-1a7a-49f7-a3e2-534a6b91dc7e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:16.039159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1de576-1a7a-49f7-a3e2-534a6b91dc7e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:16.051100 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e7391-60f4-437c-834d-61b114b08e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:16.053373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e7391-60f4-437c-834d-61b114b08e67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:16.053856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e7391-60f4-437c-834d-61b114b08e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:16.054326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e7391-60f4-437c-834d-61b114b08e67 admin admin] PATH_INFO: `/v3/projects/657459ad3f86421f85173dcdf40d8639/users/77ccdfca3efb4fb2a0d04ffc4d4181f0/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:16.077946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e7391-60f4-437c-834d-61b114b08e67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=657459ad3f86421f85173dcdf40d8639, user_id=77ccdfca3efb4fb2a0d04ffc4d4181f0, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:16.081907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e7391-60f4-437c-834d-61b114b08e67 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:16.109293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 937/1898] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:16 2022] PUT /identity/v3/projects/657459ad3f86421f85173dcdf40d8639/users/77ccdfca3efb4fb2a0d04ffc4d4181f0/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:16.113916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20024bed-88cd-43a3-8fc7-66f68c89172d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:16.125934 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20024bed-88cd-43a3-8fc7-66f68c89172d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:16.275130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:16.292607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95009513-989d-40b8-b03e-b3d65154a0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:16.295728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95009513-989d-40b8-b03e-b3d65154a0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:16.297035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95009513-989d-40b8-b03e-b3d65154a0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:16.298057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95009513-989d-40b8-b03e-b3d65154a0c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:16.300156 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95009513-989d-40b8-b03e-b3d65154a0c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:16.305157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95009513-989d-40b8-b03e-b3d65154a0c7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:16.333464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 962/1899] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:16.334955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e7391-60f4-437c-834d-61b114b08e67 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:16.356249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e7391-60f4-437c-834d-61b114b08e67 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:16.370143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9679e-ad36-481d-a993-d31f69d23db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:16.371142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9679e-ad36-481d-a993-d31f69d23db1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:16.371482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9679e-ad36-481d-a993-d31f69d23db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:16.371770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9679e-ad36-481d-a993-d31f69d23db1 admin admin] PATH_INFO: `/v3/projects/657459ad3f86421f85173dcdf40d8639/users/77ccdfca3efb4fb2a0d04ffc4d4181f0/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:16.383661 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9679e-ad36-481d-a993-d31f69d23db1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=657459ad3f86421f85173dcdf40d8639, user_id=77ccdfca3efb4fb2a0d04ffc4d4181f0, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:16.386372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9679e-ad36-481d-a993-d31f69d23db1 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:16.405331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 938/1900] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:16 2022] PUT /identity/v3/projects/657459ad3f86421f85173dcdf40d8639/users/77ccdfca3efb4fb2a0d04ffc4d4181f0/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:16.419776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65957621-71e6-4427-a385-b28c20eab8c4 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:16.419985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65957621-71e6-4427-a385-b28c20eab8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:16.420237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65957621-71e6-4427-a385-b28c20eab8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:16.475653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-65957621-71e6-4427-a385-b28c20eab8c4 None None] MFA Rules not processed for user `77ccdfca3efb4fb2a0d04ffc4d4181f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:16.477407 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-65957621-71e6-4427-a385-b28c20eab8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:16.546517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9679e-ad36-481d-a993-d31f69d23db1 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:16.548550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2dc9679e-ad36-481d-a993-d31f69d23db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:16.597235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:16.674381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9679e-ad36-481d-a993-d31f69d23db1 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:16.679049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 963/1901] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:16.690912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65957621-71e6-4427-a385-b28c20eab8c4 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:16.693922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:16.694871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:16.695148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:16.695526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] PATH_INFO: `/v3/projects/9268800faa244e33b1257fe5893cf294` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:16.698029 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9268800faa244e33b1257fe5893cf294)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:16.700123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:16.781515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9268800faa244e33b1257fe5893cf294', 'request_id': 'req-5d4df2c6-4dc5-4fac-a32b-570af126208e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:16.793131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9268800faa244e33b1257fe5893cf294', 'request_id': 'req-5d4df2c6-4dc5-4fac-a32b-570af126208e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:16.802856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9268800faa244e33b1257fe5893cf294', 'request_id': 'req-5d4df2c6-4dc5-4fac-a32b-570af126208e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:16.809743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65957621-71e6-4427-a385-b28c20eab8c4 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:16.818203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] The token cache is being invalidate because project 9268800faa244e33b1257fe5893cf294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:16.818699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:16.819403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5d4df2c6-4dc5-4fac-a32b-570af126208e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:16.821583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 939/1902] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:16 2022] DELETE /identity/v3/projects/9268800faa244e33b1257fe5893cf294 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:16.828770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7187329-aca4-4f1e-97ef-5532a9ef5588 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:16.829191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7187329-aca4-4f1e-97ef-5532a9ef5588 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:16.830247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7187329-aca4-4f1e-97ef-5532a9ef5588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:16.867114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-c7187329-aca4-4f1e-97ef-5532a9ef5588 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:16.868241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c7187329-aca4-4f1e-97ef-5532a9ef5588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:16.876200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415216e-8d22-493e-9b56-d57eb47e8451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:16.877836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415216e-8d22-493e-9b56-d57eb47e8451 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:16.878120 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415216e-8d22-493e-9b56-d57eb47e8451 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:16.878534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415216e-8d22-493e-9b56-d57eb47e8451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:16.880559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8415216e-8d22-493e-9b56-d57eb47e8451 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:16.951719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:16.961115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 940/1903] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:16.970593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231bfe7-9963-4c0f-ab6f-433dc15e1e77 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:16.970964 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231bfe7-9963-4c0f-ab6f-433dc15e1e77 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:16.971315 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231bfe7-9963-4c0f-ab6f-433dc15e1e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:16.978551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415216e-8d22-493e-9b56-d57eb47e8451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:16.980100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415216e-8d22-493e-9b56-d57eb47e8451 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:16.992180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 964/1904] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:16.995080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-0231bfe7-9963-4c0f-ab6f-433dc15e1e77 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:16.996347 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-0231bfe7-9963-4c0f-ab6f-433dc15e1e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:17.004759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 941/1905] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:17.009524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415216e-8d22-493e-9b56-d57eb47e8451 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:17.021950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415216e-8d22-493e-9b56-d57eb47e8451 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:17.026862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e37d8-eed5-46e5-b196-e3deb5dc3995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:17.027542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e37d8-eed5-46e5-b196-e3deb5dc3995 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:17.027825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e37d8-eed5-46e5-b196-e3deb5dc3995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:17.028099 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e37d8-eed5-46e5-b196-e3deb5dc3995 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:17.029246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e37d8-eed5-46e5-b196-e3deb5dc3995 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:17.029246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e37d8-eed5-46e5-b196-e3deb5dc3995 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:17.031384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e37d8-eed5-46e5-b196-e3deb5dc3995 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:17.052769 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 965/1906] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:17.058591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231bfe7-9963-4c0f-ab6f-433dc15e1e77 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:17.070819 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231bfe7-9963-4c0f-ab6f-433dc15e1e77 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:17.079378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bf87d-8147-4219-9184-8560be0a3751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:17.081016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bf87d-8147-4219-9184-8560be0a3751 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:17.081202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bf87d-8147-4219-9184-8560be0a3751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:17.081750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bf87d-8147-4219-9184-8560be0a3751 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:17.082519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5bf87d-8147-4219-9184-8560be0a3751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:17.085149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5bf87d-8147-4219-9184-8560be0a3751 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:17.105368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e37d8-eed5-46e5-b196-e3deb5dc3995 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:17.107348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1c5e37d8-eed5-46e5-b196-e3deb5dc3995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:17.121202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e37d8-eed5-46e5-b196-e3deb5dc3995 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:17.122605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 942/1907] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:17 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:17.125721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bf87d-8147-4219-9184-8560be0a3751 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:17.147961 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:17.177960 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f924e6b-b1ef-4956-88bb-4cb90f498bc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:17.178781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f924e6b-b1ef-4956-88bb-4cb90f498bc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:17.179061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f924e6b-b1ef-4956-88bb-4cb90f498bc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:17.179330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f924e6b-b1ef-4956-88bb-4cb90f498bc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:17.181156 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6f924e6b-b1ef-4956-88bb-4cb90f498bc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:17.196689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bf87d-8147-4219-9184-8560be0a3751 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:17.201450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f346ffa-c510-4b56-bd6f-a8d6f848084b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:17.202141 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346ffa-c510-4b56-bd6f-a8d6f848084b admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:17.202424 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346ffa-c510-4b56-bd6f-a8d6f848084b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:17.202739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346ffa-c510-4b56-bd6f-a8d6f848084b admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:17.203444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f346ffa-c510-4b56-bd6f-a8d6f848084b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:17.205430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f346ffa-c510-4b56-bd6f-a8d6f848084b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:17.230067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 943/1908] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:17 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:17.234243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f346ffa-c510-4b56-bd6f-a8d6f848084b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:17.244233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f346ffa-c510-4b56-bd6f-a8d6f848084b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:17.248840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fbb5a-c2a5-456d-b9ce-3d5ba65d2fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:17.249515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fbb5a-c2a5-456d-b9ce-3d5ba65d2fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:17.249795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fbb5a-c2a5-456d-b9ce-3d5ba65d2fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:17.250073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fbb5a-c2a5-456d-b9ce-3d5ba65d2fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:17.250863 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fbb5a-c2a5-456d-b9ce-3d5ba65d2fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:17.252571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fbb5a-c2a5-456d-b9ce-3d5ba65d2fe9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:17.255405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f924e6b-b1ef-4956-88bb-4cb90f498bc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:17.256981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f924e6b-b1ef-4956-88bb-4cb90f498bc2 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:17.271723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 944/1909] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:17.271910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 966/1910] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:17.273229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fbb5a-c2a5-456d-b9ce-3d5ba65d2fe9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:17.284050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fbb5a-c2a5-456d-b9ce-3d5ba65d2fe9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:17.289045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a7acd-890c-4760-819e-2044d2139bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:17.289896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a7acd-890c-4760-819e-2044d2139bad admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:17.290462 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a7acd-890c-4760-819e-2044d2139bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:17.290912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a7acd-890c-4760-819e-2044d2139bad admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:17.291791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a7acd-890c-4760-819e-2044d2139bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:17.293596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a7acd-890c-4760-819e-2044d2139bad admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:17.306721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 945/1911] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:17.310064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f924e6b-b1ef-4956-88bb-4cb90f498bc2 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:17.321595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f924e6b-b1ef-4956-88bb-4cb90f498bc2 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:17.326281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363c029-a336-4853-9281-5c8e05cbae2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:17.327229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363c029-a336-4853-9281-5c8e05cbae2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:17.327512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363c029-a336-4853-9281-5c8e05cbae2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:17.327843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363c029-a336-4853-9281-5c8e05cbae2e admin admin] PATH_INFO: `/v3/projects/fd7e3bf29f09448789adffc6a7471f22/users/2cddbc678c5a432f9fd9d8d0b0d4bc5c/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:17.344676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363c029-a336-4853-9281-5c8e05cbae2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7e3bf29f09448789adffc6a7471f22, user_id=2cddbc678c5a432f9fd9d8d0b0d4bc5c, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:17.348031 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363c029-a336-4853-9281-5c8e05cbae2e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:17.361981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 967/1912] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:17 2022] PUT /identity/v3/projects/fd7e3bf29f09448789adffc6a7471f22/users/2cddbc678c5a432f9fd9d8d0b0d4bc5c/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:17.372579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215a81-871e-43ab-bc86-08640eba981c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:17.372795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215a81-871e-43ab-bc86-08640eba981c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:17.373072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215a81-871e-43ab-bc86-08640eba981c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:17.401598 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a9215a81-871e-43ab-bc86-08640eba981c None None] MFA Rules not processed for user `2cddbc678c5a432f9fd9d8d0b0d4bc5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:17.402874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a9215a81-871e-43ab-bc86-08640eba981c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:17.488812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 946/1913] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:17.498626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363c029-a336-4853-9281-5c8e05cbae2e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:17.500965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0363c029-a336-4853-9281-5c8e05cbae2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:17.597114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363c029-a336-4853-9281-5c8e05cbae2e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:17.601133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0584e-a2c3-47d8-8edb-8aceac35463e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:17.603759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0584e-a2c3-47d8-8edb-8aceac35463e service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:17.604416 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0584e-a2c3-47d8-8edb-8aceac35463e service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:17.604530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0584e-a2c3-47d8-8edb-8aceac35463e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:17.626803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0584e-a2c3-47d8-8edb-8aceac35463e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:17.628070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0584e-a2c3-47d8-8edb-8aceac35463e service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:17.642772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 968/1914] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:22.211189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9215a81-871e-43ab-bc86-08640eba981c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:22.278767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9215a81-871e-43ab-bc86-08640eba981c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:22.283435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d085c6-a085-4bac-9d92-95070a4d01f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:22.284323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d085c6-a085-4bac-9d92-95070a4d01f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:22.284593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d085c6-a085-4bac-9d92-95070a4d01f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:22.284889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d085c6-a085-4bac-9d92-95070a4d01f3 admin admin] PATH_INFO: `/v3/users/2cddbc678c5a432f9fd9d8d0b0d4bc5c` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:22.286140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d085c6-a085-4bac-9d92-95070a4d01f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cddbc678c5a432f9fd9d8d0b0d4bc5c)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:22.288127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d085c6-a085-4bac-9d92-95070a4d01f3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:22.310932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:22.351450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-85d085c6-a085-4bac-9d92-95070a4d01f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cddbc678c5a432f9fd9d8d0b0d4bc5c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:22.361702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0584e-a2c3-47d8-8edb-8aceac35463e service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:22.363266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c4e0584e-a2c3-47d8-8edb-8aceac35463e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:22.366502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-85d085c6-a085-4bac-9d92-95070a4d01f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cddbc678c5a432f9fd9d8d0b0d4bc5c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:22.367226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-85d085c6-a085-4bac-9d92-95070a4d01f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cddbc678c5a432f9fd9d8d0b0d4bc5c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:22.367827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-85d085c6-a085-4bac-9d92-95070a4d01f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cddbc678c5a432f9fd9d8d0b0d4bc5c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:22.371917 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:22.378786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0584e-a2c3-47d8-8edb-8aceac35463e service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:22.379656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-85d085c6-a085-4bac-9d92-95070a4d01f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cddbc678c5a432f9fd9d8d0b0d4bc5c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:22.385368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf6a82-38af-454a-b0b4-d59b4cb5e715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:22.386454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf6a82-38af-454a-b0b4-d59b4cb5e715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:22.386857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf6a82-38af-454a-b0b4-d59b4cb5e715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:22.387162 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf6a82-38af-454a-b0b4-d59b4cb5e715 admin admin] PATH_INFO: `/v3/users/8fee35dd25b04cd8853a14e5ff8b44ab` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:22.388437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf6a82-38af-454a-b0b4-d59b4cb5e715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fee35dd25b04cd8853a14e5ff8b44ab)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:22.390418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf6a82-38af-454a-b0b4-d59b4cb5e715 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:22.397547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 947/1915] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:22 2022] DELETE /identity/v3/users/2cddbc678c5a432f9fd9d8d0b0d4bc5c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:22.401834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce89871-2e96-4106-84f5-723b8f7888e9 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:22.402310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce89871-2e96-4106-84f5-723b8f7888e9 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:22.402996 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce89871-2e96-4106-84f5-723b8f7888e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:22.422759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-7ce89871-2e96-4106-84f5-723b8f7888e9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:22.424932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7ce89871-2e96-4106-84f5-723b8f7888e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:22.468897 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-edcf6a82-38af-454a-b0b4-d59b4cb5e715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fee35dd25b04cd8853a14e5ff8b44ab'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:22.483880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-edcf6a82-38af-454a-b0b4-d59b4cb5e715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fee35dd25b04cd8853a14e5ff8b44ab'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:22.484916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-edcf6a82-38af-454a-b0b4-d59b4cb5e715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fee35dd25b04cd8853a14e5ff8b44ab'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:22.504019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-edcf6a82-38af-454a-b0b4-d59b4cb5e715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fee35dd25b04cd8853a14e5ff8b44ab'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:22.504904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-edcf6a82-38af-454a-b0b4-d59b4cb5e715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fee35dd25b04cd8853a14e5ff8b44ab'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:22.512758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 948/1916] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:22.526091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce89871-2e96-4106-84f5-723b8f7888e9 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:22.528419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7ce89871-2e96-4106-84f5-723b8f7888e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:22.530662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 969/1917] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:22 2022] DELETE /identity/v3/users/8fee35dd25b04cd8853a14e5ff8b44ab => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:22.536380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e719ee3-620f-46f9-b957-f3c582e218f0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:22.536758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e719ee3-620f-46f9-b957-f3c582e218f0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:22.537039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e719ee3-620f-46f9-b957-f3c582e218f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:22.560256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1e719ee3-620f-46f9-b957-f3c582e218f0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:22.561710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1e719ee3-620f-46f9-b957-f3c582e218f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:22.627921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce89871-2e96-4106-84f5-723b8f7888e9 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:22.669965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 970/1918] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:22.677674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e719ee3-620f-46f9-b957-f3c582e218f0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:22.701214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314a5dd-4e82-4624-848f-f0aabd75b6d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:22.702916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314a5dd-4e82-4624-848f-f0aabd75b6d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:22.703404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314a5dd-4e82-4624-848f-f0aabd75b6d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:22.703876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314a5dd-4e82-4624-848f-f0aabd75b6d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:22.719138 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:22.725496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314a5dd-4e82-4624-848f-f0aabd75b6d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:22.739551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314a5dd-4e82-4624-848f-f0aabd75b6d7 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:22.753058 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 949/1919] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:22.754115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314a5dd-4e82-4624-848f-f0aabd75b6d7 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:22.755771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8314a5dd-4e82-4624-848f-f0aabd75b6d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:22.828946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e719ee3-620f-46f9-b957-f3c582e218f0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:22.848505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0124a-cae3-48fd-b3d7-f0ef3425f8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:22.849259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0124a-cae3-48fd-b3d7-f0ef3425f8ca service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:22.849545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0124a-cae3-48fd-b3d7-f0ef3425f8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:22.852007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0124a-cae3-48fd-b3d7-f0ef3425f8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:22.898448 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0124a-cae3-48fd-b3d7-f0ef3425f8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:22.901386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314a5dd-4e82-4624-848f-f0aabd75b6d7 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:22.905454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0124a-cae3-48fd-b3d7-f0ef3425f8ca service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:22.906018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7072b-630b-4e60-8eaf-5a4c2abec806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:22.906781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7072b-630b-4e60-8eaf-5a4c2abec806 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:22.907066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7072b-630b-4e60-8eaf-5a4c2abec806 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:22.907343 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7072b-630b-4e60-8eaf-5a4c2abec806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:22.909836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-74b7072b-630b-4e60-8eaf-5a4c2abec806 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:22.944847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 971/1920] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:23.052647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7072b-630b-4e60-8eaf-5a4c2abec806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:23.054880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7072b-630b-4e60-8eaf-5a4c2abec806 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:23.078758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 950/1921] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:23.519082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0124a-cae3-48fd-b3d7-f0ef3425f8ca service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:23.521536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-05f0124a-cae3-48fd-b3d7-f0ef3425f8ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:23.554760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0124a-cae3-48fd-b3d7-f0ef3425f8ca service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:23.571894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:23.574770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:23.575739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:23.576033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] PATH_INFO: `/v3/projects/fd7e3bf29f09448789adffc6a7471f22` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:23.578015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7e3bf29f09448789adffc6a7471f22)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:23.591119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:23.634506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:23.706870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7072b-630b-4e60-8eaf-5a4c2abec806 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:23.708859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-74b7072b-630b-4e60-8eaf-5a4c2abec806 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:23.742379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7e3bf29f09448789adffc6a7471f22', 'request_id': 'req-bc68e33d-f4b0-4b66-a4a7-252239605b5d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:23.742992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7072b-630b-4e60-8eaf-5a4c2abec806 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:23.758834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7e3bf29f09448789adffc6a7471f22', 'request_id': 'req-bc68e33d-f4b0-4b66-a4a7-252239605b5d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:23.768250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7e3bf29f09448789adffc6a7471f22', 'request_id': 'req-bc68e33d-f4b0-4b66-a4a7-252239605b5d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:23.794768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] The token cache is being invalidate because project fd7e3bf29f09448789adffc6a7471f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:23.795151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:23.795950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc68e33d-f4b0-4b66-a4a7-252239605b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:23.799537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 972/1922] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:23 2022] DELETE /identity/v3/projects/fd7e3bf29f09448789adffc6a7471f22 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:23.805081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc991b-c001-483b-bbcc-06ff3ce7bc0e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:23.805447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc991b-c001-483b-bbcc-06ff3ce7bc0e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:23.805858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc991b-c001-483b-bbcc-06ff3ce7bc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:23.833236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-56cc991b-c001-483b-bbcc-06ff3ce7bc0e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:23.834829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-56cc991b-c001-483b-bbcc-06ff3ce7bc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:23.880230 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:23.884723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:23.886210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:23.887697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] PATH_INFO: `/v3/projects/913e817d36bc43f9aebaa92a8e7aad8c` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:23.892564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=913e817d36bc43f9aebaa92a8e7aad8c)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:23.897384 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:23.938597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 973/1923] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:23.948683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffed15-5e94-4c76-a067-b39a7532fc71 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:23.948683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffed15-5e94-4c76-a067-b39a7532fc71 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:23.949064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffed15-5e94-4c76-a067-b39a7532fc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:24.009760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-81ffed15-5e94-4c76-a067-b39a7532fc71 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:24.012662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-81ffed15-5e94-4c76-a067-b39a7532fc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:24.025752 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913e817d36bc43f9aebaa92a8e7aad8c', 'request_id': 'req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.034407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:24.043349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913e817d36bc43f9aebaa92a8e7aad8c', 'request_id': 'req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.064859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913e817d36bc43f9aebaa92a8e7aad8c', 'request_id': 'req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.093768 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] The token cache is being invalidate because project 913e817d36bc43f9aebaa92a8e7aad8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:24.094078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.094793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.101311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 951/1924] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:23 2022] DELETE /identity/v3/projects/913e817d36bc43f9aebaa92a8e7aad8c => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:24.103936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:24.106198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:24.185531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 974/1925] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:24.186794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffed15-5e94-4c76-a067-b39a7532fc71 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:24.189820 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-81ffed15-5e94-4c76-a067-b39a7532fc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:24.205386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffed15-5e94-4c76-a067-b39a7532fc71 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:24.211770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:24.213040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:24.213487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:24.213849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] PATH_INFO: `/v3/users/9418a288f693474f8fd59223b633d58c` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:24.215244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9418a288f693474f8fd59223b633d58c)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:24.217331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:24.226827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e2999-5eb6-4eda-81c8-cff7df2a0cd3 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:24.233293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be0019-3f15-4f7f-98bf-1503ecf8c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:24.234761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be0019-3f15-4f7f-98bf-1503ecf8c130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:24.235449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be0019-3f15-4f7f-98bf-1503ecf8c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:24.236273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be0019-3f15-4f7f-98bf-1503ecf8c130 admin admin] PATH_INFO: `/v3/users/77ccdfca3efb4fb2a0d04ffc4d4181f0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:24.238061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be0019-3f15-4f7f-98bf-1503ecf8c130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77ccdfca3efb4fb2a0d04ffc4d4181f0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:24.240120 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be0019-3f15-4f7f-98bf-1503ecf8c130 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:24.293686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:24.313409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9418a288f693474f8fd59223b633d58c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.353272 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9418a288f693474f8fd59223b633d58c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.354627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9418a288f693474f8fd59223b633d58c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.355238 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-86be0019-3f15-4f7f-98bf-1503ecf8c130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77ccdfca3efb4fb2a0d04ffc4d4181f0'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.373646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9418a288f693474f8fd59223b633d58c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.375236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9418a288f693474f8fd59223b633d58c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.387753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-86be0019-3f15-4f7f-98bf-1503ecf8c130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77ccdfca3efb4fb2a0d04ffc4d4181f0'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.388527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-86be0019-3f15-4f7f-98bf-1503ecf8c130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77ccdfca3efb4fb2a0d04ffc4d4181f0'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.390864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-86be0019-3f15-4f7f-98bf-1503ecf8c130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77ccdfca3efb4fb2a0d04ffc4d4181f0'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.404367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 975/1926] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:24 2022] DELETE /identity/v3/users/9418a288f693474f8fd59223b633d58c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:24.406502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:24.407390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-86be0019-3f15-4f7f-98bf-1503ecf8c130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77ccdfca3efb4fb2a0d04ffc4d4181f0'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:24.408666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:24.438499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 952/1927] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:24 2022] DELETE /identity/v3/users/77ccdfca3efb4fb2a0d04ffc4d4181f0 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:24.458404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be80fd2-eba9-4b13-ad27-f791778b16bb None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:24.460415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be80fd2-eba9-4b13-ad27-f791778b16bb None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:24.460808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be80fd2-eba9-4b13-ad27-f791778b16bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:24.506647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6be80fd2-eba9-4b13-ad27-f791778b16bb None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:24.513096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6be80fd2-eba9-4b13-ad27-f791778b16bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:24.566717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c744e-7abc-4d61-82c6-67321f855c79 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:24.613101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:24.635957 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 953/1928] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:24.651187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61059b-ec61-4249-8781-bd38dc0ab916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:24.653150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61059b-ec61-4249-8781-bd38dc0ab916 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:24.653150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61059b-ec61-4249-8781-bd38dc0ab916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:24.653150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61059b-ec61-4249-8781-bd38dc0ab916 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:24.653688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61059b-ec61-4249-8781-bd38dc0ab916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:24.654108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61059b-ec61-4249-8781-bd38dc0ab916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:24.655113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be80fd2-eba9-4b13-ad27-f791778b16bb None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:24.655880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6be80fd2-eba9-4b13-ad27-f791778b16bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:24.656209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61059b-ec61-4249-8781-bd38dc0ab916 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:24.671754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 976/1929] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:24.676450 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61059b-ec61-4249-8781-bd38dc0ab916 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:24.678338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-cc61059b-ec61-4249-8781-bd38dc0ab916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:24.694887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61059b-ec61-4249-8781-bd38dc0ab916 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:24.701115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a333d5e-b6ee-4b31-9031-ee543803f8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:24.701992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a333d5e-b6ee-4b31-9031-ee543803f8fe admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:24.702281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a333d5e-b6ee-4b31-9031-ee543803f8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:24.702646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a333d5e-b6ee-4b31-9031-ee543803f8fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:24.703610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a333d5e-b6ee-4b31-9031-ee543803f8fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:24.705780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a333d5e-b6ee-4b31-9031-ee543803f8fe admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:24.746600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 977/1930] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:24 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:24.749612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be80fd2-eba9-4b13-ad27-f791778b16bb None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:24.751439 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a333d5e-b6ee-4b31-9031-ee543803f8fe admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:24.817901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:24.858421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529ada3-c010-476d-9608-8fc150e975b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:24.859293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529ada3-c010-476d-9608-8fc150e975b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:24.859513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529ada3-c010-476d-9608-8fc150e975b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:24.859786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529ada3-c010-476d-9608-8fc150e975b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:24.886625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a333d5e-b6ee-4b31-9031-ee543803f8fe admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:24.892552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de8cab-757e-423e-85bb-62ad4f8cf155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:24.893400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de8cab-757e-423e-85bb-62ad4f8cf155 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:24.893823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de8cab-757e-423e-85bb-62ad4f8cf155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:24.895505 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de8cab-757e-423e-85bb-62ad4f8cf155 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:24.896683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de8cab-757e-423e-85bb-62ad4f8cf155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:24.899324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de8cab-757e-423e-85bb-62ad4f8cf155 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:24.928776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 978/1931] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:24 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:24.929982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de8cab-757e-423e-85bb-62ad4f8cf155 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:24.942262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529ada3-c010-476d-9608-8fc150e975b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:24.944822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529ada3-c010-476d-9608-8fc150e975b5 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:24.953775 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de8cab-757e-423e-85bb-62ad4f8cf155 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:24.960253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 954/1932] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:24.960619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55035a6-3d46-4f15-a6a2-f00f94d43d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:24.961322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55035a6-3d46-4f15-a6a2-f00f94d43d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:24.961776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55035a6-3d46-4f15-a6a2-f00f94d43d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:24.962529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55035a6-3d46-4f15-a6a2-f00f94d43d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:24.963545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55035a6-3d46-4f15-a6a2-f00f94d43d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:24.966007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55035a6-3d46-4f15-a6a2-f00f94d43d92 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:25.002873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 979/1933] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:25.004166 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529ada3-c010-476d-9608-8fc150e975b5 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:25.014765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529ada3-c010-476d-9608-8fc150e975b5 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:25.020310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55685fd7-0248-4ff7-aade-6396da676fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:25.021283 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55685fd7-0248-4ff7-aade-6396da676fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:25.021767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55685fd7-0248-4ff7-aade-6396da676fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:25.022182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55685fd7-0248-4ff7-aade-6396da676fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:25.022959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55685fd7-0248-4ff7-aade-6396da676fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:25.024675 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55685fd7-0248-4ff7-aade-6396da676fa8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:25.041178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 955/1934] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:25.046985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55035a6-3d46-4f15-a6a2-f00f94d43d92 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:25.060714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55035a6-3d46-4f15-a6a2-f00f94d43d92 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:25.067941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77184a-f061-4c75-83a9-cb33ad4bff1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:25.069034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77184a-f061-4c75-83a9-cb33ad4bff1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:25.069418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77184a-f061-4c75-83a9-cb33ad4bff1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:25.069800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77184a-f061-4c75-83a9-cb33ad4bff1c admin admin] PATH_INFO: `/v3/projects/5c61e1258f5d44ba903a0132415437c6/users/1694bd50d03f4d0f9c2ccee1b0775e12/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:25.097198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77184a-f061-4c75-83a9-cb33ad4bff1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c61e1258f5d44ba903a0132415437c6, user_id=1694bd50d03f4d0f9c2ccee1b0775e12, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:25.102135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77184a-f061-4c75-83a9-cb33ad4bff1c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:25.116008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55685fd7-0248-4ff7-aade-6396da676fa8 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:25.118028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-55685fd7-0248-4ff7-aade-6396da676fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:25.122196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 980/1935] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:25 2022] PUT /identity/v3/projects/5c61e1258f5d44ba903a0132415437c6/users/1694bd50d03f4d0f9c2ccee1b0775e12/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:25.127859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9feaa8-71b9-4fa7-bf4d-ec62d9bcd0b1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:25.128189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9feaa8-71b9-4fa7-bf4d-ec62d9bcd0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:25.128518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9feaa8-71b9-4fa7-bf4d-ec62d9bcd0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:25.136034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55685fd7-0248-4ff7-aade-6396da676fa8 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:25.164940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2c9feaa8-71b9-4fa7-bf4d-ec62d9bcd0b1 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:25.166176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2c9feaa8-71b9-4fa7-bf4d-ec62d9bcd0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:25.237698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:25.248180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:25.249084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:25.249369 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:25.249648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] PATH_INFO: `/v3/projects/657459ad3f86421f85173dcdf40d8639` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:25.250919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=657459ad3f86421f85173dcdf40d8639)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:25.253895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:25.319442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657459ad3f86421f85173dcdf40d8639', 'request_id': 'req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:25.335519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657459ad3f86421f85173dcdf40d8639', 'request_id': 'req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:25.348426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657459ad3f86421f85173dcdf40d8639', 'request_id': 'req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:25.358504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 981/1936] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:25.365671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9feaa8-71b9-4fa7-bf4d-ec62d9bcd0b1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:25.368641 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2c9feaa8-71b9-4fa7-bf4d-ec62d9bcd0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:25.368986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] The token cache is being invalidate because project 657459ad3f86421f85173dcdf40d8639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:25.369306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:25.370028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ef222f95-1b11-44d1-b3da-7fd7c6ac5d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:25.372369 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 956/1937] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:25 2022] DELETE /identity/v3/projects/657459ad3f86421f85173dcdf40d8639 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:25.379182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37ac86-3145-4e16-9f86-882ab8f76ca5 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:25.379623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37ac86-3145-4e16-9f86-882ab8f76ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:25.380553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37ac86-3145-4e16-9f86-882ab8f76ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:25.422526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-bd37ac86-3145-4e16-9f86-882ab8f76ca5 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:25.428565 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-bd37ac86-3145-4e16-9f86-882ab8f76ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:25.491634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9feaa8-71b9-4fa7-bf4d-ec62d9bcd0b1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:25.534604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 957/1938] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:25.537264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936f1e7-7c63-46ef-845d-1a083b03f8d0 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:25.537740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936f1e7-7c63-46ef-845d-1a083b03f8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:25.538119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936f1e7-7c63-46ef-845d-1a083b03f8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:25.587453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-f936f1e7-7c63-46ef-845d-1a083b03f8d0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:25.591901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:25.593845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f936f1e7-7c63-46ef-845d-1a083b03f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:25.612245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a179-b4a9-4535-b1ba-8617f49bba09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:25.612932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a179-b4a9-4535-b1ba-8617f49bba09 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:25.613223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a179-b4a9-4535-b1ba-8617f49bba09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:25.613497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a179-b4a9-4535-b1ba-8617f49bba09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:25.617467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-14d7a179-b4a9-4535-b1ba-8617f49bba09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:25.620207 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 958/1939] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:25.623914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936f1e7-7c63-46ef-845d-1a083b03f8d0 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:25.640416 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936f1e7-7c63-46ef-845d-1a083b03f8d0 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:25.654489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6747194-940d-4c4d-a441-a7d902a2969b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:25.655232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6747194-940d-4c4d-a441-a7d902a2969b admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:25.655510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6747194-940d-4c4d-a441-a7d902a2969b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:25.655785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6747194-940d-4c4d-a441-a7d902a2969b admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:25.658716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6747194-940d-4c4d-a441-a7d902a2969b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:25.659024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6747194-940d-4c4d-a441-a7d902a2969b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:25.662949 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6747194-940d-4c4d-a441-a7d902a2969b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:25.667322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7a179-b4a9-4535-b1ba-8617f49bba09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:25.669153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7a179-b4a9-4535-b1ba-8617f49bba09 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:25.690978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 982/1940] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:25.700011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 959/1941] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:59:25.701865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a179-b4a9-4535-b1ba-8617f49bba09 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:25.730972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a179-b4a9-4535-b1ba-8617f49bba09 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:25.743910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640cfa24-bd97-4fdd-97ee-3afffc95339c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:25.744529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cfa24-bd97-4fdd-97ee-3afffc95339c admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:25.744822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cfa24-bd97-4fdd-97ee-3afffc95339c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:25.745126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cfa24-bd97-4fdd-97ee-3afffc95339c admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:25.745778 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640cfa24-bd97-4fdd-97ee-3afffc95339c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:25.747722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640cfa24-bd97-4fdd-97ee-3afffc95339c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:25.809567 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 983/1942] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:25 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:25.813747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6747194-940d-4c4d-a441-a7d902a2969b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:25.828520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6747194-940d-4c4d-a441-a7d902a2969b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:25.964933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd0d6-1e87-4869-9d44-c60a67ab3535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:25.965823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd0d6-1e87-4869-9d44-c60a67ab3535 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:25.966111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd0d6-1e87-4869-9d44-c60a67ab3535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:25.966390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd0d6-1e87-4869-9d44-c60a67ab3535 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:25.967249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474dd0d6-1e87-4869-9d44-c60a67ab3535 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:25.969602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474dd0d6-1e87-4869-9d44-c60a67ab3535 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:25.983924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:26.011811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640cfa24-bd97-4fdd-97ee-3afffc95339c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:26.022622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 960/1943] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:25 2022] POST /identity/v3/users => generated 284 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:26.028508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640cfa24-bd97-4fdd-97ee-3afffc95339c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:26.035791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49377f0-0928-4d3d-8d59-b2aaa12653e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:26.037286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49377f0-0928-4d3d-8d59-b2aaa12653e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:26.037836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49377f0-0928-4d3d-8d59-b2aaa12653e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:26.038308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49377f0-0928-4d3d-8d59-b2aaa12653e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:26.039332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49377f0-0928-4d3d-8d59-b2aaa12653e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:26.041488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49377f0-0928-4d3d-8d59-b2aaa12653e9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:26.065525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd0d6-1e87-4869-9d44-c60a67ab3535 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:26.077312 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 984/1944] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:26.082539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd0d6-1e87-4869-9d44-c60a67ab3535 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:26.090103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e6793-4921-47cd-a801-044fdca6c7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:26.091612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e6793-4921-47cd-a801-044fdca6c7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:26.092078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e6793-4921-47cd-a801-044fdca6c7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:26.092570 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e6793-4921-47cd-a801-044fdca6c7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:26.093772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e6793-4921-47cd-a801-044fdca6c7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:26.096965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e6793-4921-47cd-a801-044fdca6c7c0 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:26.115377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49377f0-0928-4d3d-8d59-b2aaa12653e9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:26.121939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 961/1945] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:26.131058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49377f0-0928-4d3d-8d59-b2aaa12653e9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:26.137882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d7161-b7ac-4d01-96f7-5b987a9ce59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:26.139241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d7161-b7ac-4d01-96f7-5b987a9ce59b admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:26.139745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d7161-b7ac-4d01-96f7-5b987a9ce59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:26.140707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d7161-b7ac-4d01-96f7-5b987a9ce59b admin admin] PATH_INFO: `/v3/projects/f489c3e6d2bb4ac282ee6638ec49eba9/users/e8e3bffb29ee460e8fe71e3ee94c03f5/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:26.166015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d7161-b7ac-4d01-96f7-5b987a9ce59b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f489c3e6d2bb4ac282ee6638ec49eba9, user_id=e8e3bffb29ee460e8fe71e3ee94c03f5, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:26.174446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d7161-b7ac-4d01-96f7-5b987a9ce59b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:26.186383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e6793-4921-47cd-a801-044fdca6c7c0 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:26.188994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3e2e6793-4921-47cd-a801-044fdca6c7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:26.198625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 985/1946] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:26 2022] PUT /identity/v3/projects/f489c3e6d2bb4ac282ee6638ec49eba9/users/e8e3bffb29ee460e8fe71e3ee94c03f5/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:26.219270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebf41e-ed95-409b-a09a-de61465e287d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:26.219668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebf41e-ed95-409b-a09a-de61465e287d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:26.220007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebf41e-ed95-409b-a09a-de61465e287d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:26.225640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e6793-4921-47cd-a801-044fdca6c7c0 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:26.303404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-72ebf41e-ed95-409b-a09a-de61465e287d None None] MFA Rules not processed for user `e8e3bffb29ee460e8fe71e3ee94c03f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:26.307371 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-72ebf41e-ed95-409b-a09a-de61465e287d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:26.446466 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:26.448969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:26.449369 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:26.449833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] PATH_INFO: `/v3/projects/490f542672b14dfdb63fdecc9b7516f6` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:26.451785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=490f542672b14dfdb63fdecc9b7516f6)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:26.454535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:26.520484 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 986/1947] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:26.521197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebf41e-ed95-409b-a09a-de61465e287d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:26.524136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-72ebf41e-ed95-409b-a09a-de61465e287d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:26.575519 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebf41e-ed95-409b-a09a-de61465e287d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:26.586105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '490f542672b14dfdb63fdecc9b7516f6', 'request_id': 'req-be2de215-3707-478b-bc9e-ab6597758784'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:26.611032 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '490f542672b14dfdb63fdecc9b7516f6', 'request_id': 'req-be2de215-3707-478b-bc9e-ab6597758784'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:26.623681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '490f542672b14dfdb63fdecc9b7516f6', 'request_id': 'req-be2de215-3707-478b-bc9e-ab6597758784'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:26.649063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] The token cache is being invalidate because project 490f542672b14dfdb63fdecc9b7516f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:26.649425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:26.652041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:26.656841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 962/1948] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:26 2022] DELETE /identity/v3/projects/490f542672b14dfdb63fdecc9b7516f6 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:26.657927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:26.662825 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:26.719400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:26.721549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:26.722035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:26.722489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] PATH_INFO: `/v3/users/c5294304f59e4635ab922885295c891a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:26.727300 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5294304f59e4635ab922885295c891a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:26.735413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:26.836602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:26.868597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2de215-3707-478b-bc9e-ab6597758784 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:26.885947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e157f6-2c47-4607-8ed9-bd5ed8800a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:26.886975 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e157f6-2c47-4607-8ed9-bd5ed8800a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:26.887466 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e157f6-2c47-4607-8ed9-bd5ed8800a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:26.887892 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e157f6-2c47-4607-8ed9-bd5ed8800a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:26.888881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e157f6-2c47-4607-8ed9-bd5ed8800a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:26.891125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e157f6-2c47-4607-8ed9-bd5ed8800a05 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:26.918888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5294304f59e4635ab922885295c891a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:26.933244 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:26.945727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 963/1949] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:26 2022] POST /identity/v3/projects => generated 365 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:26.954342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e157f6-2c47-4607-8ed9-bd5ed8800a05 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:26.957226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-64e157f6-2c47-4607-8ed9-bd5ed8800a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:26.976323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5294304f59e4635ab922885295c891a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:26.977244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5294304f59e4635ab922885295c891a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:26.989882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5294304f59e4635ab922885295c891a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:26.992869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5294304f59e4635ab922885295c891a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:27.021184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 987/1950] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:26 2022] DELETE /identity/v3/users/c5294304f59e4635ab922885295c891a => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:27.021522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:27.025639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:27.109280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e157f6-2c47-4607-8ed9-bd5ed8800a05 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:27.174055 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d74ea-5161-4003-acd8-2d6bb6797769 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:27.191156 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223432a-6b60-478d-8d77-cf93ef14f3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:27.191976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223432a-6b60-478d-8d77-cf93ef14f3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:27.193334 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223432a-6b60-478d-8d77-cf93ef14f3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:27.193632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223432a-6b60-478d-8d77-cf93ef14f3a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:27.196647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223432a-6b60-478d-8d77-cf93ef14f3a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:27.201038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223432a-6b60-478d-8d77-cf93ef14f3a3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:27.264783 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:27.265794 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:27.266074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:27.268271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 988/1951] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:27 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:27.268795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] PATH_INFO: `/v3/users/a8aa25eefa4f4dd88f7d90df954e6bbb` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:27.273159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900639a2-3057-4bfe-8ca4-e589c9e6fe72 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:27.273506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900639a2-3057-4bfe-8ca4-e589c9e6fe72 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:27.273760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900639a2-3057-4bfe-8ca4-e589c9e6fe72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:27.275075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8aa25eefa4f4dd88f7d90df954e6bbb)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:27.279828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:27.314702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-900639a2-3057-4bfe-8ca4-e589c9e6fe72 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:27.321765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-900639a2-3057-4bfe-8ca4-e589c9e6fe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:27.340261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 989/1952] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:27.341342 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900639a2-3057-4bfe-8ca4-e589c9e6fe72 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:27.353484 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900639a2-3057-4bfe-8ca4-e589c9e6fe72 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:27.359621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee39b2d-06de-43fd-b181-3fae5187aaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:27.360454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39b2d-06de-43fd-b181-3fae5187aaca admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:27.360936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39b2d-06de-43fd-b181-3fae5187aaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:27.361307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee39b2d-06de-43fd-b181-3fae5187aaca admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:27.362139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee39b2d-06de-43fd-b181-3fae5187aaca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:27.364583 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee39b2d-06de-43fd-b181-3fae5187aaca admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:27.373485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8aa25eefa4f4dd88f7d90df954e6bbb'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:27.394398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8aa25eefa4f4dd88f7d90df954e6bbb'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:27.395635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8aa25eefa4f4dd88f7d90df954e6bbb'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:27.397724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8aa25eefa4f4dd88f7d90df954e6bbb'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:27.405603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 990/1953] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:27.406646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee39b2d-06de-43fd-b181-3fae5187aaca admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:27.416877 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8aa25eefa4f4dd88f7d90df954e6bbb'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:27.437330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9ee39b2d-06de-43fd-b181-3fae5187aaca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:27.459548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 964/1954] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:26 2022] DELETE /identity/v3/users/a8aa25eefa4f4dd88f7d90df954e6bbb => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:27.460570 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:27.466352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:27.534339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:27.598694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee39b2d-06de-43fd-b181-3fae5187aaca admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:27.607456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f3bcd-25e5-4793-a392-4a261919f516 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:27.613911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1de0c-2ac2-40b2-951b-7ca97bae0e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:27.615269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1de0c-2ac2-40b2-951b-7ca97bae0e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:27.615713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1de0c-2ac2-40b2-951b-7ca97bae0e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:27.616153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1de0c-2ac2-40b2-951b-7ca97bae0e6f admin admin] PATH_INFO: `/v3/projects/d8c3c59115504d33b0bda098545fc33c/users/45976f6503bd49858c2eb6594fc55c6e/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:27.639380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:27.645773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1de0c-2ac2-40b2-951b-7ca97bae0e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c3c59115504d33b0bda098545fc33c, user_id=45976f6503bd49858c2eb6594fc55c6e, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:27.648749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1de0c-2ac2-40b2-951b-7ca97bae0e6f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:27.670597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 965/1955] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:27 2022] PUT /identity/v3/projects/d8c3c59115504d33b0bda098545fc33c/users/45976f6503bd49858c2eb6594fc55c6e/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:27.675070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1de0c-2ac2-40b2-951b-7ca97bae0e6f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:27.712818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ac68f-c257-461d-9539-2e0b04dd227d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:27.713512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ac68f-c257-461d-9539-2e0b04dd227d service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:27.713799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ac68f-c257-461d-9539-2e0b04dd227d service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:27.714065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ac68f-c257-461d-9539-2e0b04dd227d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:27.715969 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ae1ac68f-c257-461d-9539-2e0b04dd227d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:27.749828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ac68f-c257-461d-9539-2e0b04dd227d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:27.752140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ac68f-c257-461d-9539-2e0b04dd227d service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:27.779921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 991/1956] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:27.781151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c8fd7-6e0b-48a5-ba85-c6691e63fd43 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:27.781440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c8fd7-6e0b-48a5-ba85-c6691e63fd43 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:27.781719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c8fd7-6e0b-48a5-ba85-c6691e63fd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:27.791719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1de0c-2ac2-40b2-951b-7ca97bae0e6f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:27.799181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a12968-3bb7-4793-a76b-b2f9b243abda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:27.800307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a12968-3bb7-4793-a76b-b2f9b243abda admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:27.800641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a12968-3bb7-4793-a76b-b2f9b243abda admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:27.800917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a12968-3bb7-4793-a76b-b2f9b243abda admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:27.802577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a12968-3bb7-4793-a76b-b2f9b243abda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:27.804314 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a12968-3bb7-4793-a76b-b2f9b243abda admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:27.817973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 966/1957] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:27.822432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a12968-3bb7-4793-a76b-b2f9b243abda admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:27.839485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a12968-3bb7-4793-a76b-b2f9b243abda admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:27.846665 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dad460-5c11-4c62-8a88-ad778814d05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:27.847928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dad460-5c11-4c62-8a88-ad778814d05a admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:27.848174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c72c8fd7-6e0b-48a5-ba85-c6691e63fd43 None None] MFA Rules not processed for user `1694bd50d03f4d0f9c2ccee1b0775e12`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:27.848280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dad460-5c11-4c62-8a88-ad778814d05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:27.848515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dad460-5c11-4c62-8a88-ad778814d05a admin admin] PATH_INFO: `/v3/projects/d8c3c59115504d33b0bda098545fc33c/users/45976f6503bd49858c2eb6594fc55c6e/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:27.850504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c72c8fd7-6e0b-48a5-ba85-c6691e63fd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:27.851915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dad460-5c11-4c62-8a88-ad778814d05a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c3c59115504d33b0bda098545fc33c, user_id=45976f6503bd49858c2eb6594fc55c6e, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:27.854874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dad460-5c11-4c62-8a88-ad778814d05a admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:27.877053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 967/1958] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:27 2022] PUT /identity/v3/projects/d8c3c59115504d33b0bda098545fc33c/users/45976f6503bd49858c2eb6594fc55c6e/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:27.889654 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f6d9-825e-4660-842b-c11d0ab05cb6 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:27.890021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f6d9-825e-4660-842b-c11d0ab05cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:27.890225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f6d9-825e-4660-842b-c11d0ab05cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:27.927868 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a563f6d9-825e-4660-842b-c11d0ab05cb6 None None] MFA Rules not processed for user `45976f6503bd49858c2eb6594fc55c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:27.929642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a563f6d9-825e-4660-842b-c11d0ab05cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:28.018372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 992/1959] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:28.030272 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c8fd7-6e0b-48a5-ba85-c6691e63fd43 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:28.032468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c72c8fd7-6e0b-48a5-ba85-c6691e63fd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:28.081201 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 968/1960] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:28.095261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563f6d9-825e-4660-842b-c11d0ab05cb6 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:28.097245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a563f6d9-825e-4660-842b-c11d0ab05cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:28.124457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:28.156799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c8fd7-6e0b-48a5-ba85-c6691e63fd43 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:28.182105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563f6d9-825e-4660-842b-c11d0ab05cb6 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:28.186478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818023d-330f-48e2-95b3-696e3902d381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:28.187194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818023d-330f-48e2-95b3-696e3902d381 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:28.187481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818023d-330f-48e2-95b3-696e3902d381 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:28.187756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818023d-330f-48e2-95b3-696e3902d381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:28.199597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818023d-330f-48e2-95b3-696e3902d381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:28.201258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818023d-330f-48e2-95b3-696e3902d381 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:28.216262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 969/1961] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:28.255608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cdf8f-8a6a-4470-9819-0c8ba72e4d97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:28.256652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cdf8f-8a6a-4470-9819-0c8ba72e4d97 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:28.257021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cdf8f-8a6a-4470-9819-0c8ba72e4d97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:28.257484 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cdf8f-8a6a-4470-9819-0c8ba72e4d97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:28.306697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:28.340912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818023d-330f-48e2-95b3-696e3902d381 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:28.353848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818023d-330f-48e2-95b3-696e3902d381 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:28.359594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2cdf8f-8a6a-4470-9819-0c8ba72e4d97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:28.361506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2cdf8f-8a6a-4470-9819-0c8ba72e4d97 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:28.364126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d50ff4-fa15-4713-8026-a0a3ba3eba31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:28.365995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d50ff4-fa15-4713-8026-a0a3ba3eba31 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:28.367635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d50ff4-fa15-4713-8026-a0a3ba3eba31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:28.369122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d50ff4-fa15-4713-8026-a0a3ba3eba31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:28.374084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 993/1962] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:28.376500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b2d50ff4-fa15-4713-8026-a0a3ba3eba31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:28.509077 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d50ff4-fa15-4713-8026-a0a3ba3eba31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:28.510686 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d50ff4-fa15-4713-8026-a0a3ba3eba31 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:28.544331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 970/1963] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:28.991110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cdf8f-8a6a-4470-9819-0c8ba72e4d97 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:28.993253 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9a2cdf8f-8a6a-4470-9819-0c8ba72e4d97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:29.160614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cdf8f-8a6a-4470-9819-0c8ba72e4d97 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:29.190078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:29.191906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:29.192375 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:29.193361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] PATH_INFO: `/v3/projects/8d2b14eeac414437a66acae944fa702b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:29.195485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d2b14eeac414437a66acae944fa702b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:29.198912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:29.307667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d2b14eeac414437a66acae944fa702b', 'request_id': 'req-baa3fa73-5a21-40bb-9c15-17ee72e977e5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.323858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d2b14eeac414437a66acae944fa702b', 'request_id': 'req-baa3fa73-5a21-40bb-9c15-17ee72e977e5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.333480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d2b14eeac414437a66acae944fa702b', 'request_id': 'req-baa3fa73-5a21-40bb-9c15-17ee72e977e5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.347559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] The token cache is being invalidate because project 8d2b14eeac414437a66acae944fa702b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:29.347856 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.348528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.350445 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 994/1964] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:28 2022] DELETE /identity/v3/projects/8d2b14eeac414437a66acae944fa702b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:29.356243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9f6ad-eeb2-4085-9ece-b2496f952ae4 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:29.356774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9f6ad-eeb2-4085-9ece-b2496f952ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:29.356774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9f6ad-eeb2-4085-9ece-b2496f952ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:29.399574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:29.401099 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:29.402440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-30a9f6ad-eeb2-4085-9ece-b2496f952ae4 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:29.403983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-30a9f6ad-eeb2-4085-9ece-b2496f952ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:29.487510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 971/1965] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:29.489792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b392a-a421-4c72-b569-cf97c0eea45a None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:29.489792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b392a-a421-4c72-b569-cf97c0eea45a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:29.490088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b392a-a421-4c72-b569-cf97c0eea45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:29.498525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3fa73-5a21-40bb-9c15-17ee72e977e5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:29.504632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:29.506178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:29.506451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:29.506779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] PATH_INFO: `/v3/projects/14b8267cc8e44486b600acc0f481076b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:29.508553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b8267cc8e44486b600acc0f481076b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:29.510534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:29.512486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:29.514404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-622b392a-a421-4c72-b569-cf97c0eea45a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:29.515963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-622b392a-a421-4c72-b569-cf97c0eea45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:29.527955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 972/1966] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:29.532140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b392a-a421-4c72-b569-cf97c0eea45a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:29.541833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b392a-a421-4c72-b569-cf97c0eea45a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:29.547082 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800d1c2-059f-4156-b4c4-521fb8238233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:29.547751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800d1c2-059f-4156-b4c4-521fb8238233 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:29.548030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800d1c2-059f-4156-b4c4-521fb8238233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:29.548299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800d1c2-059f-4156-b4c4-521fb8238233 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:29.549089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800d1c2-059f-4156-b4c4-521fb8238233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:29.549496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800d1c2-059f-4156-b4c4-521fb8238233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:29.551127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800d1c2-059f-4156-b4c4-521fb8238233 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:29.563208 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 973/1967] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:59:29.565416 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b8267cc8e44486b600acc0f481076b', 'request_id': 'req-e92d6599-baf8-4bff-bda8-ca1c3905660c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.567733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800d1c2-059f-4156-b4c4-521fb8238233 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:29.574378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b8267cc8e44486b600acc0f481076b', 'request_id': 'req-e92d6599-baf8-4bff-bda8-ca1c3905660c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.578106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800d1c2-059f-4156-b4c4-521fb8238233 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:29.580497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b8267cc8e44486b600acc0f481076b', 'request_id': 'req-e92d6599-baf8-4bff-bda8-ca1c3905660c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.592139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] The token cache is being invalidate because project 14b8267cc8e44486b600acc0f481076b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:29.592420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.593066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.594727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 995/1968] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:29 2022] DELETE /identity/v3/projects/14b8267cc8e44486b600acc0f481076b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:29.599327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:29.600636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:29.619320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:29.639048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05218c83-cf7e-4f0c-ab26-3fe3632f8e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:29.639721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05218c83-cf7e-4f0c-ab26-3fe3632f8e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:29.639996 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05218c83-cf7e-4f0c-ab26-3fe3632f8e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:29.640266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05218c83-cf7e-4f0c-ab26-3fe3632f8e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:29.641072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05218c83-cf7e-4f0c-ab26-3fe3632f8e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:29.643849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05218c83-cf7e-4f0c-ab26-3fe3632f8e73 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:29.668826 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d6599-baf8-4bff-bda8-ca1c3905660c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:29.672396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 974/1969] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:29 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:29.676466 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05218c83-cf7e-4f0c-ab26-3fe3632f8e73 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:29.676882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:29.677896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:29.678053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:29.678158 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-05218c83-cf7e-4f0c-ab26-3fe3632f8e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:29.678320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] PATH_INFO: `/v3/users/c44522bbfc7f418488a11771d8b7e38c` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:29.679592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c44522bbfc7f418488a11771d8b7e38c)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:29.681514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:29.733437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c44522bbfc7f418488a11771d8b7e38c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.745473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05218c83-cf7e-4f0c-ab26-3fe3632f8e73 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:29.746523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c44522bbfc7f418488a11771d8b7e38c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.747298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c44522bbfc7f418488a11771d8b7e38c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.750717 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b1828-7926-4a5a-ae42-be4073ec36bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:29.751618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b1828-7926-4a5a-ae42-be4073ec36bc admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:29.751947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b1828-7926-4a5a-ae42-be4073ec36bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:29.752276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b1828-7926-4a5a-ae42-be4073ec36bc admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:29.752985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b1828-7926-4a5a-ae42-be4073ec36bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:29.754976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c44522bbfc7f418488a11771d8b7e38c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.755099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b1828-7926-4a5a-ae42-be4073ec36bc admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:29.755664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c44522bbfc7f418488a11771d8b7e38c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:29.771441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 996/1970] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:29 2022] DELETE /identity/v3/users/c44522bbfc7f418488a11771d8b7e38c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:29.778755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 975/1971] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:29 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:29.783131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:29.784590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:29.795743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:29.860121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48abde-8f61-48dc-bb8d-65347642208d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:29.866010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3576d-19c9-4f83-8a63-15198cb60301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:29.866747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3576d-19c9-4f83-8a63-15198cb60301 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:29.867029 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3576d-19c9-4f83-8a63-15198cb60301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:29.867302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3576d-19c9-4f83-8a63-15198cb60301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:29.868038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3576d-19c9-4f83-8a63-15198cb60301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:29.869709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3576d-19c9-4f83-8a63-15198cb60301 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:29.881466 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 997/1972] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:29.885897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b1828-7926-4a5a-ae42-be4073ec36bc admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:29.898193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b1828-7926-4a5a-ae42-be4073ec36bc admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:29.904087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d6baa-4adf-4618-9602-ff4cb24f33ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:29.904763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d6baa-4adf-4618-9602-ff4cb24f33ef admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:29.904869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d6baa-4adf-4618-9602-ff4cb24f33ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:29.905220 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d6baa-4adf-4618-9602-ff4cb24f33ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:29.905956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d6baa-4adf-4618-9602-ff4cb24f33ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:29.907702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d6baa-4adf-4618-9602-ff4cb24f33ef admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:29.926943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 976/1973] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:29.931919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3576d-19c9-4f83-8a63-15198cb60301 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:29.943794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3576d-19c9-4f83-8a63-15198cb60301 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:29.948563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860e84-4356-498e-85bb-7c7d869f0420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:29.949464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860e84-4356-498e-85bb-7c7d869f0420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:29.949746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860e84-4356-498e-85bb-7c7d869f0420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:29.950017 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860e84-4356-498e-85bb-7c7d869f0420 admin admin] PATH_INFO: `/v3/projects/d1ed1fdb4cac499c8e9b4557d79ad7a0/users/8194af62af1140608ebf2d2a8f77672b/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:29.969947 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7860e84-4356-498e-85bb-7c7d869f0420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ed1fdb4cac499c8e9b4557d79ad7a0, user_id=8194af62af1140608ebf2d2a8f77672b, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:29.974808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7860e84-4356-498e-85bb-7c7d869f0420 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:29.993831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 998/1974] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:29 2022] PUT /identity/v3/projects/d1ed1fdb4cac499c8e9b4557d79ad7a0/users/8194af62af1140608ebf2d2a8f77672b/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:30.005486 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d05422-6b6f-4cc1-a942-5ea088fa557c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:30.005782 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d05422-6b6f-4cc1-a942-5ea088fa557c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:30.006064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d05422-6b6f-4cc1-a942-5ea088fa557c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:30.008159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5373d0-e571-4e2c-a1e3-b5d1fd436dd1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:30.008447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5373d0-e571-4e2c-a1e3-b5d1fd436dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:30.008720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5373d0-e571-4e2c-a1e3-b5d1fd436dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:30.027665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4e5373d0-e571-4e2c-a1e3-b5d1fd436dd1 None None] MFA Rules not processed for user `1694bd50d03f4d0f9c2ccee1b0775e12`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:30.028880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4e5373d0-e571-4e2c-a1e3-b5d1fd436dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:30.035532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-66d05422-6b6f-4cc1-a942-5ea088fa557c None None] MFA Rules not processed for user `8194af62af1140608ebf2d2a8f77672b`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:30.036865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-66d05422-6b6f-4cc1-a942-5ea088fa557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:30.114346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 999/1975] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:30.114346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5373d0-e571-4e2c-a1e3-b5d1fd436dd1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:30.114346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4e5373d0-e571-4e2c-a1e3-b5d1fd436dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:30.123538 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 977/1976] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:30 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:30.130547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d05422-6b6f-4cc1-a942-5ea088fa557c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:30.183857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:30.205850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5373d0-e571-4e2c-a1e3-b5d1fd436dd1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:30.213743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:30.214647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:30.214929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:30.215202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] PATH_INFO: `/v3/projects/498e6cad85d745c990012baf92f5176e` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:30.216419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=498e6cad85d745c990012baf92f5176e)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:30.218434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:30.238471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d05422-6b6f-4cc1-a942-5ea088fa557c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:30.243654 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c20fd9-1cdf-4820-8411-1d28c05f91b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:30.244337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c20fd9-1cdf-4820-8411-1d28c05f91b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:30.244621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c20fd9-1cdf-4820-8411-1d28c05f91b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:30.244890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c20fd9-1cdf-4820-8411-1d28c05f91b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:30.245589 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c20fd9-1cdf-4820-8411-1d28c05f91b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:30.247879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c20fd9-1cdf-4820-8411-1d28c05f91b0 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:30.273352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:30.285604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 978/1977] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:30 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:30.291041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c20fd9-1cdf-4820-8411-1d28c05f91b0 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:30.297898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '498e6cad85d745c990012baf92f5176e', 'request_id': 'req-2e98e542-74ac-444d-a169-ef2f3c046ab6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:30.314600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '498e6cad85d745c990012baf92f5176e', 'request_id': 'req-2e98e542-74ac-444d-a169-ef2f3c046ab6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:30.322798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '498e6cad85d745c990012baf92f5176e', 'request_id': 'req-2e98e542-74ac-444d-a169-ef2f3c046ab6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:30.344279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] The token cache is being invalidate because project 498e6cad85d745c990012baf92f5176e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:30.344638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:30.345564 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:30.347648 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1000/1978] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:30 2022] DELETE /identity/v3/projects/498e6cad85d745c990012baf92f5176e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:30.351749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:30.353752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:30.389461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c20fd9-1cdf-4820-8411-1d28c05f91b0 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:30.398308 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd67b56-0522-403e-989f-3bb09f583a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:30.399074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd67b56-0522-403e-989f-3bb09f583a70 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:30.399355 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd67b56-0522-403e-989f-3bb09f583a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:30.399631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd67b56-0522-403e-989f-3bb09f583a70 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:30.400334 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd67b56-0522-403e-989f-3bb09f583a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:30.402413 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd67b56-0522-403e-989f-3bb09f583a70 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:30.439456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 979/1979] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:30 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:30.441705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd67b56-0522-403e-989f-3bb09f583a70 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:30.445471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3dd67b56-0522-403e-989f-3bb09f583a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:30.467980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd67b56-0522-403e-989f-3bb09f583a70 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:30.473078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c6d8f-efce-4345-91bb-44d1302e99a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:30.473770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c6d8f-efce-4345-91bb-44d1302e99a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:30.474224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c6d8f-efce-4345-91bb-44d1302e99a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:30.474506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c6d8f-efce-4345-91bb-44d1302e99a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:30.475278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c6d8f-efce-4345-91bb-44d1302e99a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:30.476882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c6d8f-efce-4345-91bb-44d1302e99a1 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:30.489260 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98e542-74ac-444d-a169-ef2f3c046ab6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:30.489565 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 980/1980] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:30.493566 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c6d8f-efce-4345-91bb-44d1302e99a1 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:30.501685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:30.502619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:30.502901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:30.503174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] PATH_INFO: `/v3/users/fc4ae06588f040cfa88df9ef4d4684f3` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:30.505655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc4ae06588f040cfa88df9ef4d4684f3)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:30.507863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:30.510665 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c6d8f-efce-4345-91bb-44d1302e99a1 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:30.528720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23a390-36e7-4a88-a131-f5106ef6106f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:30.529706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23a390-36e7-4a88-a131-f5106ef6106f admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:30.530025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23a390-36e7-4a88-a131-f5106ef6106f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:30.530538 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23a390-36e7-4a88-a131-f5106ef6106f admin admin] PATH_INFO: `/v3/projects/f1759a15fe804683b9b334b373c6f3c5/users/1efc722ab840462eb25fda92ff3492ea/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:30.562172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23a390-36e7-4a88-a131-f5106ef6106f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1759a15fe804683b9b334b373c6f3c5, user_id=1efc722ab840462eb25fda92ff3492ea, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:30.565916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23a390-36e7-4a88-a131-f5106ef6106f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:30.577153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc4ae06588f040cfa88df9ef4d4684f3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:30.594446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc4ae06588f040cfa88df9ef4d4684f3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:30.595697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc4ae06588f040cfa88df9ef4d4684f3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:30.604985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 981/1981] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:30 2022] PUT /identity/v3/projects/f1759a15fe804683b9b334b373c6f3c5/users/1efc722ab840462eb25fda92ff3492ea/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:30.606086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23a390-36e7-4a88-a131-f5106ef6106f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:30.608840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5b23a390-36e7-4a88-a131-f5106ef6106f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:30.612760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc4ae06588f040cfa88df9ef4d4684f3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:30.613579 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc4ae06588f040cfa88df9ef4d4684f3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:30.641338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1001/1982] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:30 2022] DELETE /identity/v3/users/fc4ae06588f040cfa88df9ef4d4684f3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:30.716142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23a390-36e7-4a88-a131-f5106ef6106f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:30.739131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:30.813173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db745699-88ac-41cb-b11a-6ff7b143a279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:30.814622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745699-88ac-41cb-b11a-6ff7b143a279 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:30.815307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745699-88ac-41cb-b11a-6ff7b143a279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:30.815755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745699-88ac-41cb-b11a-6ff7b143a279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:30.817617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db745699-88ac-41cb-b11a-6ff7b143a279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:30.819745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db745699-88ac-41cb-b11a-6ff7b143a279 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:30.835451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 982/1983] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:30.840158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:30.854557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a67550-c49a-4b15-b114-5abee4e6907d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:30.860097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e43c17-e42c-4153-a340-03b6d178a0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:30.861331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e43c17-e42c-4153-a340-03b6d178a0c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:30.861794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e43c17-e42c-4153-a340-03b6d178a0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:30.862277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e43c17-e42c-4153-a340-03b6d178a0c0 admin admin] PATH_INFO: `/v3/projects/f1759a15fe804683b9b334b373c6f3c5/users/1efc722ab840462eb25fda92ff3492ea/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:30.864628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e43c17-e42c-4153-a340-03b6d178a0c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1759a15fe804683b9b334b373c6f3c5, user_id=1efc722ab840462eb25fda92ff3492ea, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:30.867984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e43c17-e42c-4153-a340-03b6d178a0c0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:30.890278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1002/1984] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:30 2022] PUT /identity/v3/projects/f1759a15fe804683b9b334b373c6f3c5/users/1efc722ab840462eb25fda92ff3492ea/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:30.901563 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72069ae-0a87-447b-96de-8534e6b30602 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:30.902191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72069ae-0a87-447b-96de-8534e6b30602 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:30.902542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72069ae-0a87-447b-96de-8534e6b30602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:30.960958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-c72069ae-0a87-447b-96de-8534e6b30602 None None] MFA Rules not processed for user `1efc722ab840462eb25fda92ff3492ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:30.962763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c72069ae-0a87-447b-96de-8534e6b30602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:31.014145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e43c17-e42c-4153-a340-03b6d178a0c0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:31.016279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-84e43c17-e42c-4153-a340-03b6d178a0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:31.067491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 983/1985] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:30 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:31.079043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:31.079724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72069ae-0a87-447b-96de-8534e6b30602 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:31.082044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c72069ae-0a87-447b-96de-8534e6b30602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:31.130007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e43c17-e42c-4153-a340-03b6d178a0c0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:31.140865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:31.142262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:31.142867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:31.143637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] PATH_INFO: `/v3/projects/1a55bc0976a4443f89eefb8350147124` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:31.145247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a55bc0976a4443f89eefb8350147124)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:31.147681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:31.152663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72069ae-0a87-447b-96de-8534e6b30602 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:31.156742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a37a77-05ce-4524-af9b-77a10acdb855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:31.157414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a37a77-05ce-4524-af9b-77a10acdb855 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:31.157705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a37a77-05ce-4524-af9b-77a10acdb855 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:31.158113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a37a77-05ce-4524-af9b-77a10acdb855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:31.172547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a37a77-05ce-4524-af9b-77a10acdb855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:31.174336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a37a77-05ce-4524-af9b-77a10acdb855 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:31.186535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 984/1986] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:31 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:31.204793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a55bc0976a4443f89eefb8350147124', 'request_id': 'req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:31.215338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a55bc0976a4443f89eefb8350147124', 'request_id': 'req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:31.224146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a55bc0976a4443f89eefb8350147124', 'request_id': 'req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:31.240152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] The token cache is being invalidate because project 1a55bc0976a4443f89eefb8350147124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:31.240462 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:31.241393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b429aefd-8649-4b8e-bb75-1a1ade7e26c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:31.243547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1003/1987] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:31 2022] DELETE /identity/v3/projects/1a55bc0976a4443f89eefb8350147124 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:31.250132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46286a19-52a0-4254-b482-6049b4bae118 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:31.250414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46286a19-52a0-4254-b482-6049b4bae118 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:31.250837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46286a19-52a0-4254-b482-6049b4bae118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:31.271694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-46286a19-52a0-4254-b482-6049b4bae118 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:31.273113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-46286a19-52a0-4254-b482-6049b4bae118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:31.331767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:31.356280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 985/1988] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:31.365294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bc21e-66e5-4642-b385-f95e59af8fb9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:31.365567 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bc21e-66e5-4642-b385-f95e59af8fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:31.365844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bc21e-66e5-4642-b385-f95e59af8fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:31.386796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f38bc21e-66e5-4642-b385-f95e59af8fb9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:31.388014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f38bc21e-66e5-4642-b385-f95e59af8fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:31.397984 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1004/1989] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:31.400796 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46286a19-52a0-4254-b482-6049b4bae118 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:31.411964 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46286a19-52a0-4254-b482-6049b4bae118 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:31.418205 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d0efd-db97-482f-a3e7-b19e2e47ab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:31.419137 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0efd-db97-482f-a3e7-b19e2e47ab83 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:31.419499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0efd-db97-482f-a3e7-b19e2e47ab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:31.419841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0efd-db97-482f-a3e7-b19e2e47ab83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:31.420610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d0efd-db97-482f-a3e7-b19e2e47ab83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:31.420923 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d0efd-db97-482f-a3e7-b19e2e47ab83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:31.422535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d0efd-db97-482f-a3e7-b19e2e47ab83 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:31.438445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 986/1990] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:31.441217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bc21e-66e5-4642-b385-f95e59af8fb9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:31.455139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bc21e-66e5-4642-b385-f95e59af8fb9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:31.460303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7b5d3-e004-40a4-8da4-8eef6630f548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:31.462629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7b5d3-e004-40a4-8da4-8eef6630f548 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:31.462629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7b5d3-e004-40a4-8da4-8eef6630f548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:31.462629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7b5d3-e004-40a4-8da4-8eef6630f548 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:31.463164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7b5d3-e004-40a4-8da4-8eef6630f548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:31.465173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7b5d3-e004-40a4-8da4-8eef6630f548 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:31.497914 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1005/1991] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:31 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:31.503518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d0efd-db97-482f-a3e7-b19e2e47ab83 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:31.515047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d0efd-db97-482f-a3e7-b19e2e47ab83 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:31.586297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c5454-cf36-45cd-88cf-3e9690217628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:31.587379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c5454-cf36-45cd-88cf-3e9690217628 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:31.587664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c5454-cf36-45cd-88cf-3e9690217628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:31.587938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c5454-cf36-45cd-88cf-3e9690217628 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:31.588637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c5454-cf36-45cd-88cf-3e9690217628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:31.590580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c5454-cf36-45cd-88cf-3e9690217628 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:31.613027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 987/1992] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:31 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:31.617397 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7b5d3-e004-40a4-8da4-8eef6630f548 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:31.628281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7b5d3-e004-40a4-8da4-8eef6630f548 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:31.633828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ff41c-6c81-4f93-bd1c-84f792e2d206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:31.639994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ff41c-6c81-4f93-bd1c-84f792e2d206 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:31.639994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ff41c-6c81-4f93-bd1c-84f792e2d206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:31.639994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ff41c-6c81-4f93-bd1c-84f792e2d206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:31.639994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ff41c-6c81-4f93-bd1c-84f792e2d206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:31.640318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ff41c-6c81-4f93-bd1c-84f792e2d206 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:31.656754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1006/1993] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:31.659396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c5454-cf36-45cd-88cf-3e9690217628 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:31.670191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c5454-cf36-45cd-88cf-3e9690217628 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:31.677202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950615f3-e335-401e-9b3d-9bdc7a753a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:31.678061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950615f3-e335-401e-9b3d-9bdc7a753a15 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:31.678344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950615f3-e335-401e-9b3d-9bdc7a753a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:31.678642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950615f3-e335-401e-9b3d-9bdc7a753a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:31.679376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950615f3-e335-401e-9b3d-9bdc7a753a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:31.680969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950615f3-e335-401e-9b3d-9bdc7a753a15 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:31.700841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 988/1994] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:31.705001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ff41c-6c81-4f93-bd1c-84f792e2d206 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:31.717412 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ff41c-6c81-4f93-bd1c-84f792e2d206 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:31.724117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24e2ed-ed07-4d32-92fb-8c5e96ddb022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:31.725160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e2ed-ed07-4d32-92fb-8c5e96ddb022 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:31.725503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e2ed-ed07-4d32-92fb-8c5e96ddb022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:31.725817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e2ed-ed07-4d32-92fb-8c5e96ddb022 admin admin] PATH_INFO: `/v3/projects/d230c817a7bd4a0981e3ecfcbfc03dea/users/9d188cbea43b49de835ccc1758aad8f4/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:31.743862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24e2ed-ed07-4d32-92fb-8c5e96ddb022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d230c817a7bd4a0981e3ecfcbfc03dea, user_id=9d188cbea43b49de835ccc1758aad8f4, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:31.746612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24e2ed-ed07-4d32-92fb-8c5e96ddb022 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:31.775050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1007/1995] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:31 2022] PUT /identity/v3/projects/d230c817a7bd4a0981e3ecfcbfc03dea/users/9d188cbea43b49de835ccc1758aad8f4/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:31.785871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225db97-1463-4b59-84fd-40471c562ee2 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:31.786216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225db97-1463-4b59-84fd-40471c562ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:31.786503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225db97-1463-4b59-84fd-40471c562ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:31.830605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-d225db97-1463-4b59-84fd-40471c562ee2 None None] MFA Rules not processed for user `9d188cbea43b49de835ccc1758aad8f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:31.832493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d225db97-1463-4b59-84fd-40471c562ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:31.972317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 989/1996] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:31.982691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24e2ed-ed07-4d32-92fb-8c5e96ddb022 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:32.045040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:32.065520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24e2ed-ed07-4d32-92fb-8c5e96ddb022 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:32.076782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fa757-6bcf-4b38-a593-0e71f916fb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:32.077558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fa757-6bcf-4b38-a593-0e71f916fb79 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:32.078135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fa757-6bcf-4b38-a593-0e71f916fb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:32.078482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fa757-6bcf-4b38-a593-0e71f916fb79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:32.079281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fa757-6bcf-4b38-a593-0e71f916fb79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:32.092025 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fa757-6bcf-4b38-a593-0e71f916fb79 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:32.151224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1008/1997] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:31 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:32.152308 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225db97-1463-4b59-84fd-40471c562ee2 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:32.167511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225db97-1463-4b59-84fd-40471c562ee2 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:32.247942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:32.288867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8553a72-7f5a-4cb9-b90a-fa6878bac214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:32.289779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8553a72-7f5a-4cb9-b90a-fa6878bac214 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:32.289871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8553a72-7f5a-4cb9-b90a-fa6878bac214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:32.290085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8553a72-7f5a-4cb9-b90a-fa6878bac214 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:32.290836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8553a72-7f5a-4cb9-b90a-fa6878bac214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:32.292751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8553a72-7f5a-4cb9-b90a-fa6878bac214 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:32.322807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fa757-6bcf-4b38-a593-0e71f916fb79 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:32.328391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 990/1998] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:32 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:32.337464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fa757-6bcf-4b38-a593-0e71f916fb79 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:32.348728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1be7d-a4b6-46e0-9a5c-ec7abb0f187c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:32.349430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1be7d-a4b6-46e0-9a5c-ec7abb0f187c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:32.349714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1be7d-a4b6-46e0-9a5c-ec7abb0f187c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:32.349995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1be7d-a4b6-46e0-9a5c-ec7abb0f187c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:32.354819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1be7d-a4b6-46e0-9a5c-ec7abb0f187c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:32.357216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1be7d-a4b6-46e0-9a5c-ec7abb0f187c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:32.386847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1009/1999] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:32.387948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8553a72-7f5a-4cb9-b90a-fa6878bac214 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:32.402830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8553a72-7f5a-4cb9-b90a-fa6878bac214 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:32.415397 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd55a8e-0cd0-477a-ae82-6cc7713c93f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:32.417752 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd55a8e-0cd0-477a-ae82-6cc7713c93f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:32.418051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd55a8e-0cd0-477a-ae82-6cc7713c93f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:32.418350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd55a8e-0cd0-477a-ae82-6cc7713c93f8 admin admin] PATH_INFO: `/v3/projects/b25adbc6d37a45f7a86a5fcbc0b15007/users/6297adf9ba1c4fa4a28a3dd1cda25d3f/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:32.440850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd55a8e-0cd0-477a-ae82-6cc7713c93f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25adbc6d37a45f7a86a5fcbc0b15007, user_id=6297adf9ba1c4fa4a28a3dd1cda25d3f, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:32.443833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd55a8e-0cd0-477a-ae82-6cc7713c93f8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:32.470578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 991/2000] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:32 2022] PUT /identity/v3/projects/b25adbc6d37a45f7a86a5fcbc0b15007/users/6297adf9ba1c4fa4a28a3dd1cda25d3f/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:32.470858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1be7d-a4b6-46e0-9a5c-ec7abb0f187c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:32.480962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1be7d-a4b6-46e0-9a5c-ec7abb0f187c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:32.566943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1fd27-d1dc-452b-b173-d7a1e7b2b3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:32.568234 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1fd27-d1dc-452b-b173-d7a1e7b2b3ac admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:32.568234 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1fd27-d1dc-452b-b173-d7a1e7b2b3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:32.568596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1fd27-d1dc-452b-b173-d7a1e7b2b3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:32.569436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1fd27-d1dc-452b-b173-d7a1e7b2b3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:32.571249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1fd27-d1dc-452b-b173-d7a1e7b2b3ac admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:32.584264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1010/2001] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:32.589194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd55a8e-0cd0-477a-ae82-6cc7713c93f8 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:32.606904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd55a8e-0cd0-477a-ae82-6cc7713c93f8 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:32.614993 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ed00d-3006-42cf-a587-6a7245ccba0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:32.616177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ed00d-3006-42cf-a587-6a7245ccba0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:32.616732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ed00d-3006-42cf-a587-6a7245ccba0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:32.617794 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ed00d-3006-42cf-a587-6a7245ccba0c admin admin] PATH_INFO: `/v3/projects/b25adbc6d37a45f7a86a5fcbc0b15007/users/6297adf9ba1c4fa4a28a3dd1cda25d3f/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:32.621997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ed00d-3006-42cf-a587-6a7245ccba0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25adbc6d37a45f7a86a5fcbc0b15007, user_id=6297adf9ba1c4fa4a28a3dd1cda25d3f, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:32.625603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ed00d-3006-42cf-a587-6a7245ccba0c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:32.644704 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 992/2002] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:32 2022] PUT /identity/v3/projects/b25adbc6d37a45f7a86a5fcbc0b15007/users/6297adf9ba1c4fa4a28a3dd1cda25d3f/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:32.656059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49dc38-2fd4-4326-a64e-2ab755ae93fc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:32.656588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49dc38-2fd4-4326-a64e-2ab755ae93fc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:32.656868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49dc38-2fd4-4326-a64e-2ab755ae93fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:32.694002 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-cb49dc38-2fd4-4326-a64e-2ab755ae93fc None None] MFA Rules not processed for user `6297adf9ba1c4fa4a28a3dd1cda25d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:32.696439 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-cb49dc38-2fd4-4326-a64e-2ab755ae93fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:32.801832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1011/2003] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:32.820438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ed00d-3006-42cf-a587-6a7245ccba0c admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:32.822468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5d8ed00d-3006-42cf-a587-6a7245ccba0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:32.907910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ed00d-3006-42cf-a587-6a7245ccba0c admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:32.912105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898a485-550a-433e-9038-d00e2c4f1a25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:32.912846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898a485-550a-433e-9038-d00e2c4f1a25 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:32.913274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898a485-550a-433e-9038-d00e2c4f1a25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:32.913621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898a485-550a-433e-9038-d00e2c4f1a25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:32.925131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898a485-550a-433e-9038-d00e2c4f1a25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:32.926520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898a485-550a-433e-9038-d00e2c4f1a25 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:32.937406 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 993/2004] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:33.065177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49dc38-2fd4-4326-a64e-2ab755ae93fc None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:33.073632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898a485-550a-433e-9038-d00e2c4f1a25 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:33.075106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3898a485-550a-433e-9038-d00e2c4f1a25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:33.081556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49dc38-2fd4-4326-a64e-2ab755ae93fc None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:33.090742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4024cf-f080-4ecd-a7aa-a506f74abcb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:33.093795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4024cf-f080-4ecd-a7aa-a506f74abcb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:33.094993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4024cf-f080-4ecd-a7aa-a506f74abcb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:33.095905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4024cf-f080-4ecd-a7aa-a506f74abcb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:33.192789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898a485-550a-433e-9038-d00e2c4f1a25 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:33.198343 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:33.199065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:33.199345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:33.199732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] PATH_INFO: `/v3/users/8194af62af1140608ebf2d2a8f77672b` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:33.201094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8194af62af1140608ebf2d2a8f77672b)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:33.203272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:33.210361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:33.235100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4024cf-f080-4ecd-a7aa-a506f74abcb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:33.236979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4024cf-f080-4ecd-a7aa-a506f74abcb9 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:33.253254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1012/2005] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:33.266912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:33.282285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8194af62af1140608ebf2d2a8f77672b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.302306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8194af62af1140608ebf2d2a8f77672b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.303204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8194af62af1140608ebf2d2a8f77672b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.303873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8194af62af1140608ebf2d2a8f77672b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.315376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8194af62af1140608ebf2d2a8f77672b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.332386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 994/2006] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:33 2022] DELETE /identity/v3/users/8194af62af1140608ebf2d2a8f77672b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:33.338467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d34af-262a-4b9b-b158-47dc6b8d9199 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:33.339092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d34af-262a-4b9b-b158-47dc6b8d9199 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:33.339471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d34af-262a-4b9b-b158-47dc6b8d9199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:33.368336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-712d34af-262a-4b9b-b158-47dc6b8d9199 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:33.369546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-712d34af-262a-4b9b-b158-47dc6b8d9199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:33.436528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1013/2007] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:33.446859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:33.448389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:33.519139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb997140-d047-420d-af0f-63d1780dfba2 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:33.523617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1383868-f857-4868-8fbf-7ea43874e187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:33.524445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1383868-f857-4868-8fbf-7ea43874e187 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:33.524732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1383868-f857-4868-8fbf-7ea43874e187 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:33.525033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1383868-f857-4868-8fbf-7ea43874e187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:33.535904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1383868-f857-4868-8fbf-7ea43874e187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:33.537418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1383868-f857-4868-8fbf-7ea43874e187 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:33.547494 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 995/2008] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:33.556932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d34af-262a-4b9b-b158-47dc6b8d9199 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:33.558704 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-712d34af-262a-4b9b-b158-47dc6b8d9199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:33.571710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d34af-262a-4b9b-b158-47dc6b8d9199 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:33.576541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:33.577733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:33.578340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:33.578768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] PATH_INFO: `/v3/users/9d188cbea43b49de835ccc1758aad8f4` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:33.581628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d188cbea43b49de835ccc1758aad8f4)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:33.583978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:33.643221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d188cbea43b49de835ccc1758aad8f4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.657101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d188cbea43b49de835ccc1758aad8f4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.657981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d188cbea43b49de835ccc1758aad8f4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.665885 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d188cbea43b49de835ccc1758aad8f4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.666511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d188cbea43b49de835ccc1758aad8f4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.680291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1014/2009] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:33 2022] DELETE /identity/v3/users/9d188cbea43b49de835ccc1758aad8f4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:33.685111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0401e-f0bb-4835-a4ee-a8f58c9ca40b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:33.685553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0401e-f0bb-4835-a4ee-a8f58c9ca40b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:33.685895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0401e-f0bb-4835-a4ee-a8f58c9ca40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:33.705929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-3bb0401e-f0bb-4835-a4ee-a8f58c9ca40b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:33.707333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3bb0401e-f0bb-4835-a4ee-a8f58c9ca40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:33.791109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 996/2010] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:33.804424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:33.806154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:33.877731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0401e-f0bb-4835-a4ee-a8f58c9ca40b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:33.879173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3bb0401e-f0bb-4835-a4ee-a8f58c9ca40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:33.894015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0401e-f0bb-4835-a4ee-a8f58c9ca40b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:33.897489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffcd59-ebc0-4a07-8290-7ee84cfb4a6a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:33.899774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:33.900758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:33.901052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:33.901451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] PATH_INFO: `/v3/projects/d1ed1fdb4cac499c8e9b4557d79ad7a0` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:33.902963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1ed1fdb4cac499c8e9b4557d79ad7a0)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:33.904043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e1bd9-b127-4c9c-92d6-07fc61f049c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:33.905140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:33.906190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e1bd9-b127-4c9c-92d6-07fc61f049c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:33.906909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e1bd9-b127-4c9c-92d6-07fc61f049c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:33.907248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e1bd9-b127-4c9c-92d6-07fc61f049c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:33.919125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e1bd9-b127-4c9c-92d6-07fc61f049c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:33.920859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e1bd9-b127-4c9c-92d6-07fc61f049c8 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:33.926024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:33.936287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1015/2011] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:33.975154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1ed1fdb4cac499c8e9b4557d79ad7a0', 'request_id': 'req-adf20078-1d98-4415-87ed-6eb12ae2ad52'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.979294 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:33.988897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1ed1fdb4cac499c8e9b4557d79ad7a0', 'request_id': 'req-adf20078-1d98-4415-87ed-6eb12ae2ad52'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:33.997172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1ed1fdb4cac499c8e9b4557d79ad7a0', 'request_id': 'req-adf20078-1d98-4415-87ed-6eb12ae2ad52'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.013073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] The token cache is being invalidate because project d1ed1fdb4cac499c8e9b4557d79ad7a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:34.013609 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.014322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.016156 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 997/2012] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:33 2022] DELETE /identity/v3/projects/d1ed1fdb4cac499c8e9b4557d79ad7a0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:34.020814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e1bd9-b127-4c9c-92d6-07fc61f049c8 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:34.031910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e1bd9-b127-4c9c-92d6-07fc61f049c8 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:34.117355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:34.118324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:34.118702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:34.119056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] PATH_INFO: `/v3/users/1efc722ab840462eb25fda92ff3492ea` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:34.120345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1efc722ab840462eb25fda92ff3492ea)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:34.122350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:34.212776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1efc722ab840462eb25fda92ff3492ea'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.230482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1efc722ab840462eb25fda92ff3492ea'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.231500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1efc722ab840462eb25fda92ff3492ea'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.249973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1efc722ab840462eb25fda92ff3492ea'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.250824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1efc722ab840462eb25fda92ff3492ea'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.273882 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1016/2013] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:34 2022] DELETE /identity/v3/users/1efc722ab840462eb25fda92ff3492ea => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:34.489050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:34.490685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:34.514521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf20078-1d98-4415-87ed-6eb12ae2ad52 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:34.673217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:34.674262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:34.674814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:34.675100 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] PATH_INFO: `/v3/projects/d230c817a7bd4a0981e3ecfcbfc03dea` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:34.680167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:34.682518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:34.687496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d230c817a7bd4a0981e3ecfcbfc03dea)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:34.690180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:34.720783 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31909c14-88d5-4643-83a9-711efd44871c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:34.730825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:34.736482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:34.737229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:34.737370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] PATH_INFO: `/v3/projects/f1759a15fe804683b9b334b373c6f3c5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:34.739951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1759a15fe804683b9b334b373c6f3c5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:34.742339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:34.763197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd230c817a7bd4a0981e3ecfcbfc03dea', 'request_id': 'req-60e83053-e590-4110-b4b1-93e51690ec78'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.773921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd230c817a7bd4a0981e3ecfcbfc03dea', 'request_id': 'req-60e83053-e590-4110-b4b1-93e51690ec78'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.781335 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd230c817a7bd4a0981e3ecfcbfc03dea', 'request_id': 'req-60e83053-e590-4110-b4b1-93e51690ec78'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.795227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] The token cache is being invalidate because project d230c817a7bd4a0981e3ecfcbfc03dea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:34.795525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.796229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.798966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 998/2014] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:34 2022] DELETE /identity/v3/projects/d230c817a7bd4a0981e3ecfcbfc03dea => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:34.800012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:34.802645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:34.818138 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1759a15fe804683b9b334b373c6f3c5', 'request_id': 'req-c34a04b8-9962-402d-8a05-013bcaaae808'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.834023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1759a15fe804683b9b334b373c6f3c5', 'request_id': 'req-c34a04b8-9962-402d-8a05-013bcaaae808'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.845068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1759a15fe804683b9b334b373c6f3c5', 'request_id': 'req-c34a04b8-9962-402d-8a05-013bcaaae808'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.868611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:34.876581 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] The token cache is being invalidate because project f1759a15fe804683b9b334b373c6f3c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:34.878202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.879074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:34.881416 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1017/2015] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:34 2022] DELETE /identity/v3/projects/f1759a15fe804683b9b334b373c6f3c5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:34.892429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:34.894367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:34.906040 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e83053-e590-4110-b4b1-93e51690ec78 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:34.920769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:34.973860 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125245e1-bb89-44cc-927c-6c6b3f8df421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:34.974789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125245e1-bb89-44cc-927c-6c6b3f8df421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:34.975075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125245e1-bb89-44cc-927c-6c6b3f8df421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:34.975351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125245e1-bb89-44cc-927c-6c6b3f8df421 admin admin] PATH_INFO: `/v3/users/e8e3bffb29ee460e8fe71e3ee94c03f5` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:34.976567 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125245e1-bb89-44cc-927c-6c6b3f8df421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e3bffb29ee460e8fe71e3ee94c03f5)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:34.977273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a04b8-9962-402d-8a05-013bcaaae808 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:34.978502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125245e1-bb89-44cc-927c-6c6b3f8df421 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:34.982151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80d6e7-1816-40ef-8cda-48d3e754a97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:34.983188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80d6e7-1816-40ef-8cda-48d3e754a97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:34.983570 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80d6e7-1816-40ef-8cda-48d3e754a97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:34.983955 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80d6e7-1816-40ef-8cda-48d3e754a97b admin admin] PATH_INFO: `/v3/users/6297adf9ba1c4fa4a28a3dd1cda25d3f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:34.985270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80d6e7-1816-40ef-8cda-48d3e754a97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6297adf9ba1c4fa4a28a3dd1cda25d3f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:34.987967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80d6e7-1816-40ef-8cda-48d3e754a97b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:35.031505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-125245e1-bb89-44cc-927c-6c6b3f8df421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e3bffb29ee460e8fe71e3ee94c03f5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.041267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0a80d6e7-1816-40ef-8cda-48d3e754a97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6297adf9ba1c4fa4a28a3dd1cda25d3f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.055520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-125245e1-bb89-44cc-927c-6c6b3f8df421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e3bffb29ee460e8fe71e3ee94c03f5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.058160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-125245e1-bb89-44cc-927c-6c6b3f8df421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e3bffb29ee460e8fe71e3ee94c03f5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.058891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-125245e1-bb89-44cc-927c-6c6b3f8df421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e3bffb29ee460e8fe71e3ee94c03f5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.063822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0a80d6e7-1816-40ef-8cda-48d3e754a97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6297adf9ba1c4fa4a28a3dd1cda25d3f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.066014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0a80d6e7-1816-40ef-8cda-48d3e754a97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6297adf9ba1c4fa4a28a3dd1cda25d3f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.068332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-125245e1-bb89-44cc-927c-6c6b3f8df421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e3bffb29ee460e8fe71e3ee94c03f5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.073669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0a80d6e7-1816-40ef-8cda-48d3e754a97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6297adf9ba1c4fa4a28a3dd1cda25d3f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.074357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0a80d6e7-1816-40ef-8cda-48d3e754a97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6297adf9ba1c4fa4a28a3dd1cda25d3f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.084829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 999/2016] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:34 2022] DELETE /identity/v3/users/e8e3bffb29ee460e8fe71e3ee94c03f5 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:35.086246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4292d9-354b-4d6a-963f-a9012a5ab651 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.086630 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4292d9-354b-4d6a-963f-a9012a5ab651 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.087378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4292d9-354b-4d6a-963f-a9012a5ab651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.090923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1018/2017] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:34 2022] DELETE /identity/v3/users/6297adf9ba1c4fa4a28a3dd1cda25d3f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:35.092244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a451f-1ca2-46d7-8f78-aff74ff49af8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.092531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a451f-1ca2-46d7-8f78-aff74ff49af8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.092807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a451f-1ca2-46d7-8f78-aff74ff49af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.113424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-941a451f-1ca2-46d7-8f78-aff74ff49af8 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:35.114088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-8c4292d9-354b-4d6a-963f-a9012a5ab651 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:35.114799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-941a451f-1ca2-46d7-8f78-aff74ff49af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:35.115779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8c4292d9-354b-4d6a-963f-a9012a5ab651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:35.186469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1019/2018] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:35.192929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1000/2019] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:35.199177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a451f-1ca2-46d7-8f78-aff74ff49af8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:35.200725 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-941a451f-1ca2-46d7-8f78-aff74ff49af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:35.203223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88054c-f0d6-466d-8a3d-f15942c5fe47 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.203223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88054c-f0d6-466d-8a3d-f15942c5fe47 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.203223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88054c-f0d6-466d-8a3d-f15942c5fe47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.225389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-8b88054c-f0d6-466d-8a3d-f15942c5fe47 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:35.226955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8b88054c-f0d6-466d-8a3d-f15942c5fe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:35.238667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1001/2020] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:35.240725 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:35.243127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88054c-f0d6-466d-8a3d-f15942c5fe47 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:35.255397 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88054c-f0d6-466d-8a3d-f15942c5fe47 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:35.261982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1762056-6169-4408-b4de-9bf2a742d184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:35.262938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1762056-6169-4408-b4de-9bf2a742d184 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.263230 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1762056-6169-4408-b4de-9bf2a742d184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.263993 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1762056-6169-4408-b4de-9bf2a742d184 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.264592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1762056-6169-4408-b4de-9bf2a742d184 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:35.264908 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1762056-6169-4408-b4de-9bf2a742d184 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:35.266701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1762056-6169-4408-b4de-9bf2a742d184 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:35.285288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1002/2021] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:35.286417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1762056-6169-4408-b4de-9bf2a742d184 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:35.288234 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a451f-1ca2-46d7-8f78-aff74ff49af8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:35.291638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffca98-b73d-4a48-87d5-5431a76b686e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:35.292414 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffca98-b73d-4a48-87d5-5431a76b686e service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.292761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffca98-b73d-4a48-87d5-5431a76b686e service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.293098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffca98-b73d-4a48-87d5-5431a76b686e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.297230 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1762056-6169-4408-b4de-9bf2a742d184 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:35.302460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202b248-8348-4630-a530-193b113bc480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:35.303163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202b248-8348-4630-a530-193b113bc480 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.303798 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202b248-8348-4630-a530-193b113bc480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.303798 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202b248-8348-4630-a530-193b113bc480 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.304032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffca98-b73d-4a48-87d5-5431a76b686e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:35.304422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202b248-8348-4630-a530-193b113bc480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:35.305504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffca98-b73d-4a48-87d5-5431a76b686e service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:35.307077 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202b248-8348-4630-a530-193b113bc480 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:35.321292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1020/2022] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:35.329096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:35.355442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1003/2023] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:35 2022] POST /identity/v3/projects => generated 399 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:35.357460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffca98-b73d-4a48-87d5-5431a76b686e service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:35.374224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffca98-b73d-4a48-87d5-5431a76b686e service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:35.515999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab3d6e-bd33-456a-9b2a-a65db96ea888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:35.516622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab3d6e-bd33-456a-9b2a-a65db96ea888 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.517063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab3d6e-bd33-456a-9b2a-a65db96ea888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.517839 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab3d6e-bd33-456a-9b2a-a65db96ea888 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.518918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab3d6e-bd33-456a-9b2a-a65db96ea888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:35.521537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab3d6e-bd33-456a-9b2a-a65db96ea888 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:35.563682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1021/2024] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:35 2022] POST /identity/v3/users => generated 300 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:35.565323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202b248-8348-4630-a530-193b113bc480 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:35.576336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202b248-8348-4630-a530-193b113bc480 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:35.581870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72b92f-0aff-4889-b15f-0fc716de72e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:35.582579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b92f-0aff-4889-b15f-0fc716de72e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.582874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b92f-0aff-4889-b15f-0fc716de72e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.583150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b92f-0aff-4889-b15f-0fc716de72e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.584031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72b92f-0aff-4889-b15f-0fc716de72e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:35.585668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72b92f-0aff-4889-b15f-0fc716de72e6 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:35.603606 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1004/2025] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:35.604807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab3d6e-bd33-456a-9b2a-a65db96ea888 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:35.616779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab3d6e-bd33-456a-9b2a-a65db96ea888 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:35.638329 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813526f9-37d3-4bda-972b-fbf1464f7b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:35.639109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813526f9-37d3-4bda-972b-fbf1464f7b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.639335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813526f9-37d3-4bda-972b-fbf1464f7b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.647114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813526f9-37d3-4bda-972b-fbf1464f7b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.647985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813526f9-37d3-4bda-972b-fbf1464f7b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:35.650260 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813526f9-37d3-4bda-972b-fbf1464f7b81 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:35.685634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1022/2026] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:35.689611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72b92f-0aff-4889-b15f-0fc716de72e6 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:35.701574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72b92f-0aff-4889-b15f-0fc716de72e6 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:35.710950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813526f9-37d3-4bda-972b-fbf1464f7b81 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:35.713546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-813526f9-37d3-4bda-972b-fbf1464f7b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:35.732937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1a74c-3218-4429-8ec5-3bda5012d31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:35.734113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1a74c-3218-4429-8ec5-3bda5012d31f admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.734498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1a74c-3218-4429-8ec5-3bda5012d31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.734892 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1a74c-3218-4429-8ec5-3bda5012d31f admin admin] PATH_INFO: `/v3/projects/b9e32d968dc04ca2af6681eea347172c/users/bce4538841f9436aa513d57256a705a4/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.742117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813526f9-37d3-4bda-972b-fbf1464f7b81 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:35.760456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:35.761961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.762481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.764405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] PATH_INFO: `/v3/projects/b25adbc6d37a45f7a86a5fcbc0b15007` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.766128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1a74c-3218-4429-8ec5-3bda5012d31f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e32d968dc04ca2af6681eea347172c, user_id=bce4538841f9436aa513d57256a705a4, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:35.768254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b25adbc6d37a45f7a86a5fcbc0b15007)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:35.770498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1a74c-3218-4429-8ec5-3bda5012d31f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:35.773516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:35.793687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1005/2027] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:35 2022] PUT /identity/v3/projects/b9e32d968dc04ca2af6681eea347172c/users/bce4538841f9436aa513d57256a705a4/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:35.804392 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472f39-ce00-4f20-b5b4-4660b33094ab None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.804948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472f39-ce00-4f20-b5b4-4660b33094ab None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.805975 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e472f39-ce00-4f20-b5b4-4660b33094ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.844145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2e472f39-ce00-4f20-b5b4-4660b33094ab None None] MFA Rules not processed for user `bce4538841f9436aa513d57256a705a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:35.846649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2e472f39-ce00-4f20-b5b4-4660b33094ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:35.865595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b25adbc6d37a45f7a86a5fcbc0b15007', 'request_id': 'req-1ca1c586-5b38-4391-8fb3-cf60bf0df297'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.876866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b25adbc6d37a45f7a86a5fcbc0b15007', 'request_id': 'req-1ca1c586-5b38-4391-8fb3-cf60bf0df297'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.886775 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b25adbc6d37a45f7a86a5fcbc0b15007', 'request_id': 'req-1ca1c586-5b38-4391-8fb3-cf60bf0df297'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.910582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] The token cache is being invalidate because project b25adbc6d37a45f7a86a5fcbc0b15007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:35.911066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.911926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:35.914920 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1023/2028] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:35 2022] DELETE /identity/v3/projects/b25adbc6d37a45f7a86a5fcbc0b15007 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:35.915912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:35.917790 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:35.953078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:35.978307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1006/2029] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:35.980706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01165a-6f5a-4268-99a1-3d7004ab9bb3 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:35.981613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01165a-6f5a-4268-99a1-3d7004ab9bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:35.982126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01165a-6f5a-4268-99a1-3d7004ab9bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:35.997988 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1c586-5b38-4391-8fb3-cf60bf0df297 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.002769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-ab01165a-6f5a-4268-99a1-3d7004ab9bb3 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:36.003078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.003978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.004086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ab01165a-6f5a-4268-99a1-3d7004ab9bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.004266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.004671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] PATH_INFO: `/v3/projects/f489c3e6d2bb4ac282ee6638ec49eba9` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.006491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f489c3e6d2bb4ac282ee6638ec49eba9)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.008586 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.013212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1007/2030] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:36.014177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01165a-6f5a-4268-99a1-3d7004ab9bb3 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.015700 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ab01165a-6f5a-4268-99a1-3d7004ab9bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.029890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01165a-6f5a-4268-99a1-3d7004ab9bb3 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.036225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e0ac6-4f1b-4223-a7b2-4de916ec03a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.036904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e0ac6-4f1b-4223-a7b2-4de916ec03a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.037182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e0ac6-4f1b-4223-a7b2-4de916ec03a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.037454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e0ac6-4f1b-4223-a7b2-4de916ec03a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.038157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e0ac6-4f1b-4223-a7b2-4de916ec03a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.040147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e0ac6-4f1b-4223-a7b2-4de916ec03a7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.070881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1008/2031] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:36 2022] POST /identity/v3/projects => generated 397 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:36.070881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e55c7-a1b5-4b2a-8377-3284cfe356e6 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.070881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e55c7-a1b5-4b2a-8377-3284cfe356e6 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.070881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e55c7-a1b5-4b2a-8377-3284cfe356e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.074413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f489c3e6d2bb4ac282ee6638ec49eba9', 'request_id': 'req-974c7728-72b2-46b9-b71c-c77f7930d615'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.082866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f489c3e6d2bb4ac282ee6638ec49eba9', 'request_id': 'req-974c7728-72b2-46b9-b71c-c77f7930d615'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.089276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f489c3e6d2bb4ac282ee6638ec49eba9', 'request_id': 'req-974c7728-72b2-46b9-b71c-c77f7930d615'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.090150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-183e55c7-a1b5-4b2a-8377-3284cfe356e6 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:36.091410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-183e55c7-a1b5-4b2a-8377-3284cfe356e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.101110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] The token cache is being invalidate because project f489c3e6d2bb4ac282ee6638ec49eba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:36.101391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.102050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.103700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1024/2032] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:35 2022] DELETE /identity/v3/projects/f489c3e6d2bb4ac282ee6638ec49eba9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:36.106695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.106695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.156389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1009/2033] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:36.157381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e55c7-a1b5-4b2a-8377-3284cfe356e6 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.159125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-183e55c7-a1b5-4b2a-8377-3284cfe356e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.171180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e55c7-a1b5-4b2a-8377-3284cfe356e6 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.176310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.177202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.177475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.177741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] PATH_INFO: `/v3/users/45976f6503bd49858c2eb6594fc55c6e` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.178857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c7728-72b2-46b9-b71c-c77f7930d615 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.178972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=45976f6503bd49858c2eb6594fc55c6e)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.180889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.184168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bec478-452d-45d7-ad66-de0fe5a04093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.184864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bec478-452d-45d7-ad66-de0fe5a04093 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.185144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bec478-452d-45d7-ad66-de0fe5a04093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.185424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bec478-452d-45d7-ad66-de0fe5a04093 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.186128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bec478-452d-45d7-ad66-de0fe5a04093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.188114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bec478-452d-45d7-ad66-de0fe5a04093 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.198880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:36.214398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1025/2034] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:36 2022] POST /identity/v3/users => generated 305 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:36.214398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bec478-452d-45d7-ad66-de0fe5a04093 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.225703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bec478-452d-45d7-ad66-de0fe5a04093 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.230528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fa486-e87f-4b38-a8ac-8ce16075d6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.231230 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fa486-e87f-4b38-a8ac-8ce16075d6ca admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.231500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fa486-e87f-4b38-a8ac-8ce16075d6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.231774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fa486-e87f-4b38-a8ac-8ce16075d6ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.232511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fa486-e87f-4b38-a8ac-8ce16075d6ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:36.232825 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fa486-e87f-4b38-a8ac-8ce16075d6ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.234443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fa486-e87f-4b38-a8ac-8ce16075d6ca admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.238740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45976f6503bd49858c2eb6594fc55c6e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.247548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1026/2035] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:36.248551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fa486-e87f-4b38-a8ac-8ce16075d6ca admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.251863 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45976f6503bd49858c2eb6594fc55c6e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.252900 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45976f6503bd49858c2eb6594fc55c6e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.253451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45976f6503bd49858c2eb6594fc55c6e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.259112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fa486-e87f-4b38-a8ac-8ce16075d6ca admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.261116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45976f6503bd49858c2eb6594fc55c6e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.264378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c80c7-9e3e-4f01-88e0-8e7d832ed053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.265293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c80c7-9e3e-4f01-88e0-8e7d832ed053 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.265408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c80c7-9e3e-4f01-88e0-8e7d832ed053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.265618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c80c7-9e3e-4f01-88e0-8e7d832ed053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.266350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c80c7-9e3e-4f01-88e0-8e7d832ed053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.268109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c80c7-9e3e-4f01-88e0-8e7d832ed053 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.276265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1010/2036] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:36 2022] DELETE /identity/v3/users/45976f6503bd49858c2eb6594fc55c6e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:36.279274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.279274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.279651 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1027/2037] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:36.283895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c80c7-9e3e-4f01-88e0-8e7d832ed053 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.285998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5d3c80c7-9e3e-4f01-88e0-8e7d832ed053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.300923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c80c7-9e3e-4f01-88e0-8e7d832ed053 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.318666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:36.359322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a11a43-5f3b-4e44-b416-46f78e57f0fd admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.365036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268c355-24a0-4be3-a7e8-b84f9f085f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.365727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268c355-24a0-4be3-a7e8-b84f9f085f04 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.366021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268c355-24a0-4be3-a7e8-b84f9f085f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.366290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268c355-24a0-4be3-a7e8-b84f9f085f04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.367037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268c355-24a0-4be3-a7e8-b84f9f085f04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.368971 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268c355-24a0-4be3-a7e8-b84f9f085f04 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.371567 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b3fae-384d-4385-abe7-1ce4c7ad750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.372831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b3fae-384d-4385-abe7-1ce4c7ad750f admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.373384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b3fae-384d-4385-abe7-1ce4c7ad750f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.374509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b3fae-384d-4385-abe7-1ce4c7ad750f admin admin] PATH_INFO: `/v3/projects/8ee11b1a9c4e4af88ff861660c23d870/users/98e0d20414464277877de2deaec26244/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.398803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1011/2038] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:36 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:36.402867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268c355-24a0-4be3-a7e8-b84f9f085f04 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.408204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b3fae-384d-4385-abe7-1ce4c7ad750f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee11b1a9c4e4af88ff861660c23d870, user_id=98e0d20414464277877de2deaec26244, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.410911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b3fae-384d-4385-abe7-1ce4c7ad750f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.428684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1028/2039] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:36 2022] PUT /identity/v3/projects/8ee11b1a9c4e4af88ff861660c23d870/users/98e0d20414464277877de2deaec26244/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:36.433019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b3fae-384d-4385-abe7-1ce4c7ad750f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.481853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268c355-24a0-4be3-a7e8-b84f9f085f04 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.511468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:36.535698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b3fae-384d-4385-abe7-1ce4c7ad750f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.541312 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a16b6-c70e-4e1b-8ba6-a4e4a761edba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.542027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a16b6-c70e-4e1b-8ba6-a4e4a761edba admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.542381 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a16b6-c70e-4e1b-8ba6-a4e4a761edba admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.542763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a16b6-c70e-4e1b-8ba6-a4e4a761edba admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.543587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a16b6-c70e-4e1b-8ba6-a4e4a761edba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.545348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a16b6-c70e-4e1b-8ba6-a4e4a761edba admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.558184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1029/2040] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:36.562337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c03cd-5956-4e69-930c-0f50ecff2e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.562946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a16b6-c70e-4e1b-8ba6-a4e4a761edba admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.563236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c03cd-5956-4e69-930c-0f50ecff2e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.563581 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c03cd-5956-4e69-930c-0f50ecff2e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.563916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c03cd-5956-4e69-930c-0f50ecff2e2d admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.565241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c03cd-5956-4e69-930c-0f50ecff2e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.567481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c03cd-5956-4e69-930c-0f50ecff2e2d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.575363 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a16b6-c70e-4e1b-8ba6-a4e4a761edba admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.581621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15629c-f9f0-4830-8c6a-37324d21be79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.582437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15629c-f9f0-4830-8c6a-37324d21be79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.582884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15629c-f9f0-4830-8c6a-37324d21be79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.583254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15629c-f9f0-4830-8c6a-37324d21be79 admin admin] PATH_INFO: `/v3/projects/8ee11b1a9c4e4af88ff861660c23d870/users/98e0d20414464277877de2deaec26244/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.586362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15629c-f9f0-4830-8c6a-37324d21be79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee11b1a9c4e4af88ff861660c23d870, user_id=98e0d20414464277877de2deaec26244, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.589606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15629c-f9f0-4830-8c6a-37324d21be79 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.600801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1012/2041] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:36 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:36.603323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c03cd-5956-4e69-930c-0f50ecff2e2d admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.607115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1030/2042] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:36 2022] PUT /identity/v3/projects/8ee11b1a9c4e4af88ff861660c23d870/users/98e0d20414464277877de2deaec26244/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:36.615119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c03cd-5956-4e69-930c-0f50ecff2e2d admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.620569 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40caf3-5c16-427a-89dc-e9257767ca08 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.620887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40caf3-5c16-427a-89dc-e9257767ca08 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.621204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40caf3-5c16-427a-89dc-e9257767ca08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.661174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7c40caf3-5c16-427a-89dc-e9257767ca08 None None] MFA Rules not processed for user `98e0d20414464277877de2deaec26244`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:36.662994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7c40caf3-5c16-427a-89dc-e9257767ca08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.706872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c449a27-08bf-4f4f-ab80-29920565e669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.707762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c449a27-08bf-4f4f-ab80-29920565e669 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.708043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c449a27-08bf-4f4f-ab80-29920565e669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.708313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c449a27-08bf-4f4f-ab80-29920565e669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.709042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c449a27-08bf-4f4f-ab80-29920565e669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.710697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c449a27-08bf-4f4f-ab80-29920565e669 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.726365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1013/2043] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:36.726365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c449a27-08bf-4f4f-ab80-29920565e669 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.728205 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6c449a27-08bf-4f4f-ab80-29920565e669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.741762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c449a27-08bf-4f4f-ab80-29920565e669 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.746480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.747395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.747674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.747950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] PATH_INFO: `/v3/projects/d8c3c59115504d33b0bda098545fc33c` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.749327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c3c59115504d33b0bda098545fc33c)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.751359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.757602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1031/2044] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:36.762108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40caf3-5c16-427a-89dc-e9257767ca08 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.773688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40caf3-5c16-427a-89dc-e9257767ca08 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.778633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635046d-7f20-41ee-864f-6af48e1ef6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.779118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635046d-7f20-41ee-864f-6af48e1ef6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.779476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635046d-7f20-41ee-864f-6af48e1ef6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.779821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635046d-7f20-41ee-864f-6af48e1ef6e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.780701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635046d-7f20-41ee-864f-6af48e1ef6e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.782373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635046d-7f20-41ee-864f-6af48e1ef6e1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.790104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:36.800174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1032/2045] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:36.801614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635046d-7f20-41ee-864f-6af48e1ef6e1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.804132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2635046d-7f20-41ee-864f-6af48e1ef6e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.828307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c3c59115504d33b0bda098545fc33c', 'request_id': 'req-de3364a6-9c66-4ae8-b763-90ef2ef8a774'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.837836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c3c59115504d33b0bda098545fc33c', 'request_id': 'req-de3364a6-9c66-4ae8-b763-90ef2ef8a774'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.844821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c3c59115504d33b0bda098545fc33c', 'request_id': 'req-de3364a6-9c66-4ae8-b763-90ef2ef8a774'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.863794 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] The token cache is being invalidate because project d8c3c59115504d33b0bda098545fc33c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:36.864088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.864789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:36.866419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1014/2046] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:36 2022] DELETE /identity/v3/projects/d8c3c59115504d33b0bda098545fc33c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:36.867352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:36.868735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.879678 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635046d-7f20-41ee-864f-6af48e1ef6e1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.917763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:36.940116 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2098c3-f49a-48fa-8881-40cfb51dd098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.940931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2098c3-f49a-48fa-8881-40cfb51dd098 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.941201 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2098c3-f49a-48fa-8881-40cfb51dd098 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.941468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2098c3-f49a-48fa-8881-40cfb51dd098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.943527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dd2098c3-f49a-48fa-8881-40cfb51dd098 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:36.948928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3364a6-9c66-4ae8-b763-90ef2ef8a774 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:36.953888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c86d5-bd06-4acd-beaa-1190b6d9dc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:36.954931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c86d5-bd06-4acd-beaa-1190b6d9dc0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.955256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c86d5-bd06-4acd-beaa-1190b6d9dc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.955559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c86d5-bd06-4acd-beaa-1190b6d9dc0f admin admin] PATH_INFO: `/v3/projects/8862565d7ca44216b7dc76e8df01f1cd/users/3dcd7a5d240f4e46ac5edd0a6126cbbd/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:36.974767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c86d5-bd06-4acd-beaa-1190b6d9dc0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8862565d7ca44216b7dc76e8df01f1cd, user_id=3dcd7a5d240f4e46ac5edd0a6126cbbd, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:36.977378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c86d5-bd06-4acd-beaa-1190b6d9dc0f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:36.992790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1015/2047] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:36 2022] PUT /identity/v3/projects/8862565d7ca44216b7dc76e8df01f1cd/users/3dcd7a5d240f4e46ac5edd0a6126cbbd/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:36.994018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e8f9-08fb-4f6c-9cee-584206df73d0 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:36.994302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e8f9-08fb-4f6c-9cee-584206df73d0 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:36.994588 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e8f9-08fb-4f6c-9cee-584206df73d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.014439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-4f17e8f9-08fb-4f6c-9cee-584206df73d0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:37.016297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4f17e8f9-08fb-4f6c-9cee-584206df73d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:37.035698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2098c3-f49a-48fa-8881-40cfb51dd098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:37.037343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2098c3-f49a-48fa-8881-40cfb51dd098 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:37.057114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1033/2048] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:37.059384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37005c-7070-4107-810a-c546519dccff None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.063118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37005c-7070-4107-810a-c546519dccff None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.063510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37005c-7070-4107-810a-c546519dccff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.119416 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4e37005c-7070-4107-810a-c546519dccff None None] MFA Rules not processed for user `3dcd7a5d240f4e46ac5edd0a6126cbbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:37.121889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1016/2049] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:37.123236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4e37005c-7070-4107-810a-c546519dccff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:37.130923 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e597b39-00af-4b43-8da7-133e1f39a104 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.131228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e597b39-00af-4b43-8da7-133e1f39a104 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.131478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e597b39-00af-4b43-8da7-133e1f39a104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.171569 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-4e597b39-00af-4b43-8da7-133e1f39a104 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:37.172856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4e597b39-00af-4b43-8da7-133e1f39a104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:37.194993 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1017/2050] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:37.195997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e597b39-00af-4b43-8da7-133e1f39a104 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:37.197459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4e597b39-00af-4b43-8da7-133e1f39a104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:37.217837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:37.236027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1034/2051] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:37.236027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37005c-7070-4107-810a-c546519dccff None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:37.245173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37005c-7070-4107-810a-c546519dccff None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:37.250116 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243467-ab21-46be-a20e-c7e29cbe3be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:37.250954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243467-ab21-46be-a20e-c7e29cbe3be3 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.251236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243467-ab21-46be-a20e-c7e29cbe3be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.251545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243467-ab21-46be-a20e-c7e29cbe3be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.252290 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc243467-ab21-46be-a20e-c7e29cbe3be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:37.253035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc243467-ab21-46be-a20e-c7e29cbe3be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:37.254615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc243467-ab21-46be-a20e-c7e29cbe3be3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:37.273565 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1035/2052] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:37.273565 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243467-ab21-46be-a20e-c7e29cbe3be3 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:37.276600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dc243467-ab21-46be-a20e-c7e29cbe3be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:37.286879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e597b39-00af-4b43-8da7-133e1f39a104 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:37.290790 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a354421-f4a4-46d4-a8b3-f4de6595966b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:37.291460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354421-f4a4-46d4-a8b3-f4de6595966b service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.291936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354421-f4a4-46d4-a8b3-f4de6595966b service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.292065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354421-f4a4-46d4-a8b3-f4de6595966b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.292880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243467-ab21-46be-a20e-c7e29cbe3be3 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:37.294266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8a354421-f4a4-46d4-a8b3-f4de6595966b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:37.298428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c690a-d269-429c-bcf5-bfedcb3100f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:37.299285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c690a-d269-429c-bcf5-bfedcb3100f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.299655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c690a-d269-429c-bcf5-bfedcb3100f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.300145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c690a-d269-429c-bcf5-bfedcb3100f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.310037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c690a-d269-429c-bcf5-bfedcb3100f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:37.311653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c690a-d269-429c-bcf5-bfedcb3100f1 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:37.321111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1036/2053] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:37.322001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c690a-d269-429c-bcf5-bfedcb3100f1 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:37.334855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c690a-d269-429c-bcf5-bfedcb3100f1 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:37.342146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad48c6-2fe9-4237-9181-07b5f2ba1601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:37.342936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad48c6-2fe9-4237-9181-07b5f2ba1601 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.343341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad48c6-2fe9-4237-9181-07b5f2ba1601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.343692 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad48c6-2fe9-4237-9181-07b5f2ba1601 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.344456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad48c6-2fe9-4237-9181-07b5f2ba1601 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:37.346722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad48c6-2fe9-4237-9181-07b5f2ba1601 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:37.359648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a354421-f4a4-46d4-a8b3-f4de6595966b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:37.362039 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a354421-f4a4-46d4-a8b3-f4de6595966b service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:37.374271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1018/2054] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:37.379038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1037/2055] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:37 2022] POST /identity/v3/projects => generated 361 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:37.383112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a354421-f4a4-46d4-a8b3-f4de6595966b service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:37.394283 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a354421-f4a4-46d4-a8b3-f4de6595966b service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:37.398194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:37.464196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26459f-c140-495c-bb27-22331ee4c9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:37.465104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26459f-c140-495c-bb27-22331ee4c9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.465478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26459f-c140-495c-bb27-22331ee4c9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.465840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26459f-c140-495c-bb27-22331ee4c9a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.466644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26459f-c140-495c-bb27-22331ee4c9a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:37.468873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26459f-c140-495c-bb27-22331ee4c9a8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:37.495752 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1019/2056] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:37 2022] POST /identity/v3/users => generated 281 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:37.497140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad48c6-2fe9-4237-9181-07b5f2ba1601 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:37.510376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad48c6-2fe9-4237-9181-07b5f2ba1601 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:37.516517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6fed5-5ad1-48c8-898a-20878c768a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:37.517466 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6fed5-5ad1-48c8-898a-20878c768a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.517684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6fed5-5ad1-48c8-898a-20878c768a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.518080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6fed5-5ad1-48c8-898a-20878c768a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.519487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6fed5-5ad1-48c8-898a-20878c768a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:37.521081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6fed5-5ad1-48c8-898a-20878c768a87 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:37.534005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1038/2057] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:37.540176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26459f-c140-495c-bb27-22331ee4c9a8 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:37.552091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26459f-c140-495c-bb27-22331ee4c9a8 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:37.557578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ede93-c679-4a4b-92c1-611161ec42af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:37.558738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ede93-c679-4a4b-92c1-611161ec42af admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.559055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ede93-c679-4a4b-92c1-611161ec42af admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.559536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ede93-c679-4a4b-92c1-611161ec42af admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.560228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ede93-c679-4a4b-92c1-611161ec42af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:37.562079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ede93-c679-4a4b-92c1-611161ec42af admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:37.573879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1020/2058] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:37.578242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6fed5-5ad1-48c8-898a-20878c768a87 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:37.590286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6fed5-5ad1-48c8-898a-20878c768a87 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:37.595787 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187acc96-191d-4adc-8dc0-1eb1cd02f45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:37.596735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187acc96-191d-4adc-8dc0-1eb1cd02f45b admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.597058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187acc96-191d-4adc-8dc0-1eb1cd02f45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.597384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187acc96-191d-4adc-8dc0-1eb1cd02f45b admin admin] PATH_INFO: `/v3/projects/ab664580acd2425ab63ec250a92c64ed/users/3955e78f4d57478096e65b65cf86eb60/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.615842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187acc96-191d-4adc-8dc0-1eb1cd02f45b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab664580acd2425ab63ec250a92c64ed, user_id=3955e78f4d57478096e65b65cf86eb60, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:37.618841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187acc96-191d-4adc-8dc0-1eb1cd02f45b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:37.643794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1039/2059] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:37 2022] PUT /identity/v3/projects/ab664580acd2425ab63ec250a92c64ed/users/3955e78f4d57478096e65b65cf86eb60/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:37.664311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d651c5-fd02-4220-a99a-40b9b992fb55 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.664605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d651c5-fd02-4220-a99a-40b9b992fb55 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.664983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d651c5-fd02-4220-a99a-40b9b992fb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.720077 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-00d651c5-fd02-4220-a99a-40b9b992fb55 None None] MFA Rules not processed for user `3955e78f4d57478096e65b65cf86eb60`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:37.721902 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-00d651c5-fd02-4220-a99a-40b9b992fb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:37.797341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1021/2060] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:37.834266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187acc96-191d-4adc-8dc0-1eb1cd02f45b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:37.911185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187acc96-191d-4adc-8dc0-1eb1cd02f45b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:37.917108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c51922-1b13-459a-a446-4313d403e523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:37.917891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c51922-1b13-459a-a446-4313d403e523 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:37.918265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c51922-1b13-459a-a446-4313d403e523 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:37.918665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c51922-1b13-459a-a446-4313d403e523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:37.934178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c51922-1b13-459a-a446-4313d403e523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:37.936329 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c51922-1b13-459a-a446-4313d403e523 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:37.948115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1040/2061] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:37.963925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d651c5-fd02-4220-a99a-40b9b992fb55 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:38.042180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d651c5-fd02-4220-a99a-40b9b992fb55 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:38.047446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:38.048421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:38.048874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:38.049655 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] PATH_INFO: `/v3/users/3955e78f4d57478096e65b65cf86eb60` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:38.050951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3955e78f4d57478096e65b65cf86eb60)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:38.052873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:38.071644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:38.104829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3955e78f4d57478096e65b65cf86eb60'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:38.118772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3955e78f4d57478096e65b65cf86eb60'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:38.119715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3955e78f4d57478096e65b65cf86eb60'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:38.120339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3955e78f4d57478096e65b65cf86eb60'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:38.127833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3955e78f4d57478096e65b65cf86eb60'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:38.146545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1022/2062] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:37 2022] DELETE /identity/v3/users/3955e78f4d57478096e65b65cf86eb60 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:38.151328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd92218-cb60-446d-b25d-9c05a93419ff None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:38.151680 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd92218-cb60-446d-b25d-9c05a93419ff None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:38.151957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd92218-cb60-446d-b25d-9c05a93419ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:38.171296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5bd92218-cb60-446d-b25d-9c05a93419ff None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:38.172568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5bd92218-cb60-446d-b25d-9c05a93419ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:38.242975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1041/2063] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:38.251711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:38.253050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:38.325063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300135b9-1ce6-4903-acef-bb8b5dcbfde9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:38.328956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3f107-dc98-4721-b753-3b3b98bc3b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:38.329755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3f107-dc98-4721-b753-3b3b98bc3b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:38.330205 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3f107-dc98-4721-b753-3b3b98bc3b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:38.330543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3f107-dc98-4721-b753-3b3b98bc3b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:38.343557 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3f107-dc98-4721-b753-3b3b98bc3b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:38.345021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3f107-dc98-4721-b753-3b3b98bc3b39 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:38.356457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1023/2064] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:38.790728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd92218-cb60-446d-b25d-9c05a93419ff None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:38.792227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5bd92218-cb60-446d-b25d-9c05a93419ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:38.806916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd92218-cb60-446d-b25d-9c05a93419ff None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:38.829008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:38.832538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:38.832821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:38.833095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] PATH_INFO: `/v3/projects/ab664580acd2425ab63ec250a92c64ed` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:38.837876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab664580acd2425ab63ec250a92c64ed)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:38.840012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:38.873164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:38.935257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab664580acd2425ab63ec250a92c64ed', 'request_id': 'req-7e0289de-9883-409d-9ddf-4207fbd2daa3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:38.951622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab664580acd2425ab63ec250a92c64ed', 'request_id': 'req-7e0289de-9883-409d-9ddf-4207fbd2daa3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:38.963819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab664580acd2425ab63ec250a92c64ed', 'request_id': 'req-7e0289de-9883-409d-9ddf-4207fbd2daa3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:38.981763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] The token cache is being invalidate because project ab664580acd2425ab63ec250a92c64ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:38.982054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:38.982926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7e0289de-9883-409d-9ddf-4207fbd2daa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:38.985019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1042/2065] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:38 2022] DELETE /identity/v3/projects/ab664580acd2425ab63ec250a92c64ed => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:38.992094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c2bb5-6d74-4613-95be-d7a5fb845f43 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:38.992568 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c2bb5-6d74-4613-95be-d7a5fb845f43 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:38.992660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c2bb5-6d74-4613-95be-d7a5fb845f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:39.020452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-053c2bb5-6d74-4613-95be-d7a5fb845f43 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:39.022280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-053c2bb5-6d74-4613-95be-d7a5fb845f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:39.120374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1024/2066] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:39.129245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224e04b-2320-406e-9eda-8038b9a2f47f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:39.129530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224e04b-2320-406e-9eda-8038b9a2f47f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:39.129812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224e04b-2320-406e-9eda-8038b9a2f47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:39.152298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7224e04b-2320-406e-9eda-8038b9a2f47f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:39.153748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7224e04b-2320-406e-9eda-8038b9a2f47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:39.166472 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1043/2067] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:39.171160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c2bb5-6d74-4613-95be-d7a5fb845f43 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:39.181370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c2bb5-6d74-4613-95be-d7a5fb845f43 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:39.187318 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f026e0-7d67-4eaf-a154-8320b9d680e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:39.188019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f026e0-7d67-4eaf-a154-8320b9d680e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:39.188301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f026e0-7d67-4eaf-a154-8320b9d680e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:39.188577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f026e0-7d67-4eaf-a154-8320b9d680e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:39.189361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f026e0-7d67-4eaf-a154-8320b9d680e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:39.189669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f026e0-7d67-4eaf-a154-8320b9d680e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:39.191370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f026e0-7d67-4eaf-a154-8320b9d680e5 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:39.210090 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1025/2068] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:39.211300 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224e04b-2320-406e-9eda-8038b9a2f47f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:39.227204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224e04b-2320-406e-9eda-8038b9a2f47f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:39.234600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb52c58-1f8f-4274-ae85-769619805eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:39.236455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52c58-1f8f-4274-ae85-769619805eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:39.236833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52c58-1f8f-4274-ae85-769619805eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:39.237318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52c58-1f8f-4274-ae85-769619805eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:39.244447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb52c58-1f8f-4274-ae85-769619805eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:39.247816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb52c58-1f8f-4274-ae85-769619805eb8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:39.290622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1044/2069] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:39 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:39.293087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f026e0-7d67-4eaf-a154-8320b9d680e5 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:39.304379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f026e0-7d67-4eaf-a154-8320b9d680e5 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:39.327819 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:39.370991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22258e5-e157-4d84-99c3-5a44b3da459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:39.371683 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22258e5-e157-4d84-99c3-5a44b3da459c admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:39.371960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22258e5-e157-4d84-99c3-5a44b3da459c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:39.372287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22258e5-e157-4d84-99c3-5a44b3da459c admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:39.372940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22258e5-e157-4d84-99c3-5a44b3da459c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:39.374946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22258e5-e157-4d84-99c3-5a44b3da459c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:39.397151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1026/2070] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:39 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:39.401566 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb52c58-1f8f-4274-ae85-769619805eb8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:39.412963 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb52c58-1f8f-4274-ae85-769619805eb8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:39.417789 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cabb4-c535-4a25-9c9a-0ef7fd1010b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:39.418532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cabb4-c535-4a25-9c9a-0ef7fd1010b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:39.419015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cabb4-c535-4a25-9c9a-0ef7fd1010b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:39.419293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cabb4-c535-4a25-9c9a-0ef7fd1010b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:39.420028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cabb4-c535-4a25-9c9a-0ef7fd1010b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:39.421804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cabb4-c535-4a25-9c9a-0ef7fd1010b5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:39.434742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1045/2071] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:39.438923 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22258e5-e157-4d84-99c3-5a44b3da459c admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:39.450647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22258e5-e157-4d84-99c3-5a44b3da459c admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:39.457734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d4c56-e7aa-4a15-9033-cb9fa13ab5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:39.458414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d4c56-e7aa-4a15-9033-cb9fa13ab5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:39.458732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d4c56-e7aa-4a15-9033-cb9fa13ab5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:39.458998 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d4c56-e7aa-4a15-9033-cb9fa13ab5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:39.459719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d4c56-e7aa-4a15-9033-cb9fa13ab5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:39.462205 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d4c56-e7aa-4a15-9033-cb9fa13ab5d1 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:39.484153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1027/2072] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:39.489677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cabb4-c535-4a25-9c9a-0ef7fd1010b5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:39.503280 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cabb4-c535-4a25-9c9a-0ef7fd1010b5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:39.509107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb1590-44fd-460c-b65c-e20dd20c18b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:39.510251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb1590-44fd-460c-b65c-e20dd20c18b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:39.510734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb1590-44fd-460c-b65c-e20dd20c18b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:39.511160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb1590-44fd-460c-b65c-e20dd20c18b0 admin admin] PATH_INFO: `/v3/projects/0192e4102b644993a9482674674b1ad7/users/48ff681e601f4892abffe5862def0547/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:39.529615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb1590-44fd-460c-b65c-e20dd20c18b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0192e4102b644993a9482674674b1ad7, user_id=48ff681e601f4892abffe5862def0547, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:39.532421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb1590-44fd-460c-b65c-e20dd20c18b0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:39.555908 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1046/2073] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:39 2022] PUT /identity/v3/projects/0192e4102b644993a9482674674b1ad7/users/48ff681e601f4892abffe5862def0547/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:39.557234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c72cd6-c82b-4ed7-b9c4-7537d79fbc4d None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:39.557521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c72cd6-c82b-4ed7-b9c4-7537d79fbc4d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:39.557856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c72cd6-c82b-4ed7-b9c4-7537d79fbc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:39.581616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-41c72cd6-c82b-4ed7-b9c4-7537d79fbc4d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:39.583173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-41c72cd6-c82b-4ed7-b9c4-7537d79fbc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:39.669964 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1028/2074] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:39.685712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb1590-44fd-460c-b65c-e20dd20c18b0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:39.687498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-04fb1590-44fd-460c-b65c-e20dd20c18b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:39.755079 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb1590-44fd-460c-b65c-e20dd20c18b0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:39.759118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1052f-96f6-4ebe-8009-d89be2a4d4dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:39.759855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1052f-96f6-4ebe-8009-d89be2a4d4dc service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:39.760197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1052f-96f6-4ebe-8009-d89be2a4d4dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:39.760583 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1052f-96f6-4ebe-8009-d89be2a4d4dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:39.772657 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1052f-96f6-4ebe-8009-d89be2a4d4dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:39.774146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1052f-96f6-4ebe-8009-d89be2a4d4dc service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:39.785878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1047/2075] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:43.025355 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c72cd6-c82b-4ed7-b9c4-7537d79fbc4d None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:43.027254 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-41c72cd6-c82b-4ed7-b9c4-7537d79fbc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:43.041320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c72cd6-c82b-4ed7-b9c4-7537d79fbc4d None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:43.047327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:43.048230 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:43.048575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:43.048850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] PATH_INFO: `/v3/users/bce4538841f9436aa513d57256a705a4` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:43.055925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bce4538841f9436aa513d57256a705a4)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:43.057924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:43.148317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bce4538841f9436aa513d57256a705a4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:43.187402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bce4538841f9436aa513d57256a705a4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:43.191226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bce4538841f9436aa513d57256a705a4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:43.191984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bce4538841f9436aa513d57256a705a4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:43.218947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bce4538841f9436aa513d57256a705a4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:43.248630 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1029/2076] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:43 2022] DELETE /identity/v3/users/bce4538841f9436aa513d57256a705a4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:43.253160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4432f-22d6-47e9-8499-de63b41cebb8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:43.254027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4432f-22d6-47e9-8499-de63b41cebb8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:43.254508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4432f-22d6-47e9-8499-de63b41cebb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:43.283219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0ed4432f-22d6-47e9-8499-de63b41cebb8 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:43.284904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0ed4432f-22d6-47e9-8499-de63b41cebb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:43.404054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:43.479646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1048/2077] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:43.495156 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:43.496992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:43.553583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:43.622307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018c142-20e9-4f2c-90ba-21ca34f51b95 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:43.630637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231c5bc-63f7-484d-9377-cbe04891337e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:43.633120 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231c5bc-63f7-484d-9377-cbe04891337e service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:43.634009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231c5bc-63f7-484d-9377-cbe04891337e service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:43.634409 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231c5bc-63f7-484d-9377-cbe04891337e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:43.651317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231c5bc-63f7-484d-9377-cbe04891337e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:43.652880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231c5bc-63f7-484d-9377-cbe04891337e service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:43.684761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1030/2078] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:44.550377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4432f-22d6-47e9-8499-de63b41cebb8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:44.551973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0ed4432f-22d6-47e9-8499-de63b41cebb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:44.577059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4432f-22d6-47e9-8499-de63b41cebb8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:44.583405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:44.584700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:44.585133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:44.585593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] PATH_INFO: `/v3/projects/b9e32d968dc04ca2af6681eea347172c` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:44.587524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e32d968dc04ca2af6681eea347172c)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:44.590515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:44.680253 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e32d968dc04ca2af6681eea347172c', 'request_id': 'req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:44.697275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e32d968dc04ca2af6681eea347172c', 'request_id': 'req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:44.715465 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e32d968dc04ca2af6681eea347172c', 'request_id': 'req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:44.759461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] The token cache is being invalidate because project b9e32d968dc04ca2af6681eea347172c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:44.759762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:44.761170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:44.768872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1049/2079] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:44 2022] DELETE /identity/v3/projects/b9e32d968dc04ca2af6681eea347172c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:44.769831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231c5bc-63f7-484d-9377-cbe04891337e service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:44.794362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231c5bc-63f7-484d-9377-cbe04891337e service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:44.867226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35399a3-cebd-4aa2-a032-65931d855126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:44.868219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35399a3-cebd-4aa2-a032-65931d855126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:44.868552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35399a3-cebd-4aa2-a032-65931d855126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:44.868893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35399a3-cebd-4aa2-a032-65931d855126 admin admin] PATH_INFO: `/v3/users/98e0d20414464277877de2deaec26244` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:44.870741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35399a3-cebd-4aa2-a032-65931d855126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e0d20414464277877de2deaec26244)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:44.872902 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35399a3-cebd-4aa2-a032-65931d855126 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:44.976326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a35399a3-cebd-4aa2-a032-65931d855126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e0d20414464277877de2deaec26244'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:44.990113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a35399a3-cebd-4aa2-a032-65931d855126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e0d20414464277877de2deaec26244'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:44.991108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a35399a3-cebd-4aa2-a032-65931d855126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e0d20414464277877de2deaec26244'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:44.991869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a35399a3-cebd-4aa2-a032-65931d855126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e0d20414464277877de2deaec26244'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:45.002362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a35399a3-cebd-4aa2-a032-65931d855126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e0d20414464277877de2deaec26244'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:45.033775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1031/2080] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:44 2022] DELETE /identity/v3/users/98e0d20414464277877de2deaec26244 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:45.458326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:45.461858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:45.503558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e169d7-acb1-4bf2-8f1a-41ff6b94e87e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:45.550708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1a10d-99cb-4c8f-acb2-f638895a12ed None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:45.551251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1a10d-99cb-4c8f-acb2-f638895a12ed None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:45.551534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1a10d-99cb-4c8f-acb2-f638895a12ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:45.602143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:45.603240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:45.603585 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:45.603912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] PATH_INFO: `/v3/projects/8ee11b1a9c4e4af88ff861660c23d870` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:45.604281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2af1a10d-99cb-4c8f-acb2-f638895a12ed None None] MFA Rules not processed for user `48ff681e601f4892abffe5862def0547`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:45.605512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2af1a10d-99cb-4c8f-acb2-f638895a12ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:45.605666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ee11b1a9c4e4af88ff861660c23d870)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:45.607652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:45.640019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:45.668803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ee11b1a9c4e4af88ff861660c23d870', 'request_id': 'req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:45.678203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ee11b1a9c4e4af88ff861660c23d870', 'request_id': 'req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:45.684610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:45.686985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ee11b1a9c4e4af88ff861660c23d870', 'request_id': 'req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:45.697572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1032/2081] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:45.703086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1a10d-99cb-4c8f-acb2-f638895a12ed None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:45.704933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] The token cache is being invalidate because project 8ee11b1a9c4e4af88ff861660c23d870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:45.705069 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2af1a10d-99cb-4c8f-acb2-f638895a12ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:45.705320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:45.706253 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5ca3b7a2-25b7-4f25-b9ba-af1cc4a2ab72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:45.708175 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1050/2082] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:45 2022] DELETE /identity/v3/projects/8ee11b1a9c4e4af88ff861660c23d870 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:45.713853 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84dbbf-adb0-482b-9550-176342a31aad None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:45.714198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84dbbf-adb0-482b-9550-176342a31aad None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:45.714833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84dbbf-adb0-482b-9550-176342a31aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:45.724045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1a10d-99cb-4c8f-acb2-f638895a12ed None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:45.737609 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dd84dbbf-adb0-482b-9550-176342a31aad None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:45.738959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dd84dbbf-adb0-482b-9550-176342a31aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:45.799379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d01f1-b75d-417d-b6dd-3520a83630ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:45.800151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d01f1-b75d-417d-b6dd-3520a83630ff admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:45.800525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d01f1-b75d-417d-b6dd-3520a83630ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:45.800862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d01f1-b75d-417d-b6dd-3520a83630ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:45.802247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d01f1-b75d-417d-b6dd-3520a83630ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:45.803797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d01f1-b75d-417d-b6dd-3520a83630ff admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:45.813838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1051/2083] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:45.824807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c46ff-59fe-49f5-b6d9-7be9a887ee3b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:45.825455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c46ff-59fe-49f5-b6d9-7be9a887ee3b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:45.825747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c46ff-59fe-49f5-b6d9-7be9a887ee3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:45.839576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1033/2084] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:45 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:45.843406 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d01f1-b75d-417d-b6dd-3520a83630ff admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:45.845664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f07d01f1-b75d-417d-b6dd-3520a83630ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:45.847326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-738c46ff-59fe-49f5-b6d9-7be9a887ee3b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:45.848489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-738c46ff-59fe-49f5-b6d9-7be9a887ee3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:45.925461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1052/2085] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:45.927560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d01f1-b75d-417d-b6dd-3520a83630ff admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:45.930341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c46ff-59fe-49f5-b6d9-7be9a887ee3b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:45.932777 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73986ac-9c35-4a6e-bc95-85be49a53443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:45.933494 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73986ac-9c35-4a6e-bc95-85be49a53443 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:45.933777 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73986ac-9c35-4a6e-bc95-85be49a53443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:45.934053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73986ac-9c35-4a6e-bc95-85be49a53443 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:45.935030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73986ac-9c35-4a6e-bc95-85be49a53443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:45.937563 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73986ac-9c35-4a6e-bc95-85be49a53443 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:45.940822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c46ff-59fe-49f5-b6d9-7be9a887ee3b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:45.946753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c83b8-3a49-4676-8c95-9e2962932b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:45.947898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c83b8-3a49-4676-8c95-9e2962932b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:45.948393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c83b8-3a49-4676-8c95-9e2962932b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:45.948845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c83b8-3a49-4676-8c95-9e2962932b38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:45.949859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c83b8-3a49-4676-8c95-9e2962932b38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:45.950265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c83b8-3a49-4676-8c95-9e2962932b38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:45.952664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c83b8-3a49-4676-8c95-9e2962932b38 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:45.967027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1053/2086] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 09:59:45.971245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1034/2087] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:45 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:45.971404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c83b8-3a49-4676-8c95-9e2962932b38 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:45.975367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73986ac-9c35-4a6e-bc95-85be49a53443 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:45.983065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c83b8-3a49-4676-8c95-9e2962932b38 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:45.986408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73986ac-9c35-4a6e-bc95-85be49a53443 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:45.988634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785054e0-c5bc-4d65-884c-ceb45695e050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:45.989329 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785054e0-c5bc-4d65-884c-ceb45695e050 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:45.989632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785054e0-c5bc-4d65-884c-ceb45695e050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:45.989942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785054e0-c5bc-4d65-884c-ceb45695e050 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:45.990691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785054e0-c5bc-4d65-884c-ceb45695e050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:45.992671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785054e0-c5bc-4d65-884c-ceb45695e050 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:45.994084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d6cdc-3def-4029-bd30-769431021d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:45.994865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d6cdc-3def-4029-bd30-769431021d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:45.995213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d6cdc-3def-4029-bd30-769431021d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:45.995555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d6cdc-3def-4029-bd30-769431021d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:45.996364 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d6cdc-3def-4029-bd30-769431021d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:45.998044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d6cdc-3def-4029-bd30-769431021d13 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:46.009583 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1035/2088] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:46.013615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d6cdc-3def-4029-bd30-769431021d13 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:46.025970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d6cdc-3def-4029-bd30-769431021d13 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:46.030677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1054/2089] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:45 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:46.033005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785054e0-c5bc-4d65-884c-ceb45695e050 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:46.033984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a3071-4421-465b-96ee-1da8d1717ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:46.034938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a3071-4421-465b-96ee-1da8d1717ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:46.035223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a3071-4421-465b-96ee-1da8d1717ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:46.035525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a3071-4421-465b-96ee-1da8d1717ca2 admin admin] PATH_INFO: `/v3/projects/eb2108eb54074b92896b5f0fe77ecf3f/users/5a2d88a23143416083a691af22d70a66/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:46.058270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a3071-4421-465b-96ee-1da8d1717ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb2108eb54074b92896b5f0fe77ecf3f, user_id=5a2d88a23143416083a691af22d70a66, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:46.061165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a3071-4421-465b-96ee-1da8d1717ca2 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:46.082322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1036/2090] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:46 2022] PUT /identity/v3/projects/eb2108eb54074b92896b5f0fe77ecf3f/users/5a2d88a23143416083a691af22d70a66/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:46.085855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a3071-4421-465b-96ee-1da8d1717ca2 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:46.097533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:46.110490 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:46.115120 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785054e0-c5bc-4d65-884c-ceb45695e050 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:46.218034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81c9ac-dd77-4894-bb1e-1136966c3003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:46.219694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81c9ac-dd77-4894-bb1e-1136966c3003 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:46.220032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81c9ac-dd77-4894-bb1e-1136966c3003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:46.221072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81c9ac-dd77-4894-bb1e-1136966c3003 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:46.222219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81c9ac-dd77-4894-bb1e-1136966c3003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:46.224130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81c9ac-dd77-4894-bb1e-1136966c3003 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:46.230338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a3071-4421-465b-96ee-1da8d1717ca2 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:46.238188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114d346-6789-4826-a976-0955a47bd159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:46.239361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114d346-6789-4826-a976-0955a47bd159 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:46.239636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114d346-6789-4826-a976-0955a47bd159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:46.240043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114d346-6789-4826-a976-0955a47bd159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:46.240664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114d346-6789-4826-a976-0955a47bd159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:46.242475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114d346-6789-4826-a976-0955a47bd159 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:46.254611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1055/2091] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:46 2022] POST /identity/v3/users => generated 287 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:46.259691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81c9ac-dd77-4894-bb1e-1136966c3003 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:46.264146 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1037/2092] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:46.265789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114d346-6789-4826-a976-0955a47bd159 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:46.272073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81c9ac-dd77-4894-bb1e-1136966c3003 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:46.281226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2567e-5703-4440-96fe-d7b794965871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:46.282223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2567e-5703-4440-96fe-d7b794965871 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:46.282362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2567e-5703-4440-96fe-d7b794965871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:46.283464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2567e-5703-4440-96fe-d7b794965871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:46.283810 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2567e-5703-4440-96fe-d7b794965871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:46.285938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2567e-5703-4440-96fe-d7b794965871 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:46.289049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114d346-6789-4826-a976-0955a47bd159 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:46.296329 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5468c7-4221-4c5c-89a6-8d92fd9f442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:46.297247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5468c7-4221-4c5c-89a6-8d92fd9f442a admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:46.297529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5468c7-4221-4c5c-89a6-8d92fd9f442a admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:46.298605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5468c7-4221-4c5c-89a6-8d92fd9f442a admin admin] PATH_INFO: `/v3/projects/eb2108eb54074b92896b5f0fe77ecf3f/users/5a2d88a23143416083a691af22d70a66/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:46.300515 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5468c7-4221-4c5c-89a6-8d92fd9f442a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb2108eb54074b92896b5f0fe77ecf3f, user_id=5a2d88a23143416083a691af22d70a66, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:46.304616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5468c7-4221-4c5c-89a6-8d92fd9f442a admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:46.325893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1056/2093] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:46.326076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1038/2094] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:46 2022] PUT /identity/v3/projects/eb2108eb54074b92896b5f0fe77ecf3f/users/5a2d88a23143416083a691af22d70a66/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:46.326942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2567e-5703-4440-96fe-d7b794965871 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:46.339164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2567e-5703-4440-96fe-d7b794965871 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:46.422976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453143d-14d8-4a21-8fff-41aa6a3ac5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:46.423460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453143d-14d8-4a21-8fff-41aa6a3ac5aa admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:46.424281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453143d-14d8-4a21-8fff-41aa6a3ac5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:46.424281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453143d-14d8-4a21-8fff-41aa6a3ac5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:46.424823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453143d-14d8-4a21-8fff-41aa6a3ac5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:46.426670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453143d-14d8-4a21-8fff-41aa6a3ac5aa admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:46.443401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1057/2095] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:46.448246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5468c7-4221-4c5c-89a6-8d92fd9f442a admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:46.470885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5468c7-4221-4c5c-89a6-8d92fd9f442a admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:46.479178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e5c3d-427b-4249-9a6a-ead41cd7b047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:46.480185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e5c3d-427b-4249-9a6a-ead41cd7b047 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:46.480688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e5c3d-427b-4249-9a6a-ead41cd7b047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:46.480998 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e5c3d-427b-4249-9a6a-ead41cd7b047 admin admin] PATH_INFO: `/v3/projects/8c1f850b21dd4b048311c653945ddd55/users/8f35faab058042198dde279a7429265b/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:46.506546 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e5c3d-427b-4249-9a6a-ead41cd7b047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c1f850b21dd4b048311c653945ddd55, user_id=8f35faab058042198dde279a7429265b, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:46.511495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e5c3d-427b-4249-9a6a-ead41cd7b047 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:46.534769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1039/2096] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:46 2022] PUT /identity/v3/projects/8c1f850b21dd4b048311c653945ddd55/users/8f35faab058042198dde279a7429265b/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:46.540979 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4562eac-e5b6-438b-910e-faf19be7e82f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:46.541489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4562eac-e5b6-438b-910e-faf19be7e82f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:46.541922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4562eac-e5b6-438b-910e-faf19be7e82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:46.565596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b4562eac-e5b6-438b-910e-faf19be7e82f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:46.567333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b4562eac-e5b6-438b-910e-faf19be7e82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:46.588067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e5c3d-427b-4249-9a6a-ead41cd7b047 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:46.589952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f93e5c3d-427b-4249-9a6a-ead41cd7b047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:46.714909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1058/2097] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:46.731673 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4562eac-e5b6-438b-910e-faf19be7e82f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:46.735491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b4562eac-e5b6-438b-910e-faf19be7e82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:46.744850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e5c3d-427b-4249-9a6a-ead41cd7b047 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:46.760159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:46.760820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:46.761100 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:46.761374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] PATH_INFO: `/v3/users/3dcd7a5d240f4e46ac5edd0a6126cbbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:46.762636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dcd7a5d240f4e46ac5edd0a6126cbbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:46.763924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:46.767272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:46.833879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4562eac-e5b6-438b-910e-faf19be7e82f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:46.839044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36de99-75f5-4497-b427-6d23062e2ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:46.839763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36de99-75f5-4497-b427-6d23062e2ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:46.840041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36de99-75f5-4497-b427-6d23062e2ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:46.840371 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36de99-75f5-4497-b427-6d23062e2ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:46.856040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36de99-75f5-4497-b427-6d23062e2ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:46.858747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36de99-75f5-4497-b427-6d23062e2ee7 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:46.872071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1059/2098] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:46.879047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dcd7a5d240f4e46ac5edd0a6126cbbd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:46.907236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dcd7a5d240f4e46ac5edd0a6126cbbd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:46.908056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dcd7a5d240f4e46ac5edd0a6126cbbd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:46.908685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dcd7a5d240f4e46ac5edd0a6126cbbd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:46.915510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:46.929204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dcd7a5d240f4e46ac5edd0a6126cbbd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:46.956519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1040/2099] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:46 2022] DELETE /identity/v3/users/3dcd7a5d240f4e46ac5edd0a6126cbbd => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 09:59:46.959034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b42e1-ac6a-45e1-8f96-7992a13d2f97 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:46.959507 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b42e1-ac6a-45e1-8f96-7992a13d2f97 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:46.959924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b42e1-ac6a-45e1-8f96-7992a13d2f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:47.010965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fc7b42e1-ac6a-45e1-8f96-7992a13d2f97 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:47.012288 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fc7b42e1-ac6a-45e1-8f96-7992a13d2f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:47.150530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1060/2100] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:47.173789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:47.175489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:47.292297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ffe86-53b4-44c5-8ac1-08d55de1f66e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:47.296789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1f0bf-dc06-435c-a74d-509969b08e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:47.297750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1f0bf-dc06-435c-a74d-509969b08e9a service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:47.298472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1f0bf-dc06-435c-a74d-509969b08e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:47.299301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1f0bf-dc06-435c-a74d-509969b08e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:47.320445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1f0bf-dc06-435c-a74d-509969b08e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:47.322699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1f0bf-dc06-435c-a74d-509969b08e9a service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:47.339494 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1041/2101] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:47.837725 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c47d-2da6-480a-bde2-a6fee02aae0e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:47.838287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c47d-2da6-480a-bde2-a6fee02aae0e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:47.838287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4c47d-2da6-480a-bde2-a6fee02aae0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:47.878263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b7a4c47d-2da6-480a-bde2-a6fee02aae0e None None] MFA Rules not processed for user `8f35faab058042198dde279a7429265b`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:47.879612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b7a4c47d-2da6-480a-bde2-a6fee02aae0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:47.973131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1061/2102] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:47.977863 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1f0bf-dc06-435c-a74d-509969b08e9a service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:47.984621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4bf1f0bf-dc06-435c-a74d-509969b08e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:48.004363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1f0bf-dc06-435c-a74d-509969b08e9a service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:48.011255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94f2cd-b217-4a63-96cc-fdce0aea9dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:48.012106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94f2cd-b217-4a63-96cc-fdce0aea9dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:48.012389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94f2cd-b217-4a63-96cc-fdce0aea9dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:48.012663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94f2cd-b217-4a63-96cc-fdce0aea9dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:48.013394 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94f2cd-b217-4a63-96cc-fdce0aea9dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:48.016096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94f2cd-b217-4a63-96cc-fdce0aea9dd2 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:48.074638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1042/2103] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:47 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:48.076725 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4c47d-2da6-480a-bde2-a6fee02aae0e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:48.095562 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4c47d-2da6-480a-bde2-a6fee02aae0e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:48.195432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:48.211361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e7e1d-3712-495e-82ff-014664d30b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:48.212494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e7e1d-3712-495e-82ff-014664d30b3e admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:48.212835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e7e1d-3712-495e-82ff-014664d30b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:48.213173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e7e1d-3712-495e-82ff-014664d30b3e admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:48.213972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e7e1d-3712-495e-82ff-014664d30b3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:48.216945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e7e1d-3712-495e-82ff-014664d30b3e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:48.269438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1062/2104] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:48 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:48.270756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94f2cd-b217-4a63-96cc-fdce0aea9dd2 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:48.283327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94f2cd-b217-4a63-96cc-fdce0aea9dd2 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:48.303217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85c5ee-11fd-48a7-98f4-778b2d7684be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:48.303929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85c5ee-11fd-48a7-98f4-778b2d7684be admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:48.304212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85c5ee-11fd-48a7-98f4-778b2d7684be admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:48.304491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85c5ee-11fd-48a7-98f4-778b2d7684be admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:48.305240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85c5ee-11fd-48a7-98f4-778b2d7684be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:48.312132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85c5ee-11fd-48a7-98f4-778b2d7684be admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:48.350165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1043/2105] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:48.353562 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e7e1d-3712-495e-82ff-014664d30b3e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:48.380011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e7e1d-3712-495e-82ff-014664d30b3e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:48.392088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dfb8c-a53b-43b0-95e2-97f8351d0cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:48.394147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dfb8c-a53b-43b0-95e2-97f8351d0cf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:48.394443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dfb8c-a53b-43b0-95e2-97f8351d0cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:48.394753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dfb8c-a53b-43b0-95e2-97f8351d0cf5 admin admin] PATH_INFO: `/v3/projects/9d0e7518ffb1418ea24b1507d229f3c4/users/360d89d785164e0f9b8be6795339482a/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:48.399760 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85c5ee-11fd-48a7-98f4-778b2d7684be admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:48.401566 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7e85c5ee-11fd-48a7-98f4-778b2d7684be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:48.419758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7dfb8c-a53b-43b0-95e2-97f8351d0cf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0e7518ffb1418ea24b1507d229f3c4, user_id=360d89d785164e0f9b8be6795339482a, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:48.422747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7dfb8c-a53b-43b0-95e2-97f8351d0cf5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:48.425049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85c5ee-11fd-48a7-98f4-778b2d7684be admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:48.435934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:48.437990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:48.438278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:48.439218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] PATH_INFO: `/v3/projects/8862565d7ca44216b7dc76e8df01f1cd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:48.443423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8862565d7ca44216b7dc76e8df01f1cd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:48.447736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:48.456446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1063/2106] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:48 2022] PUT /identity/v3/projects/9d0e7518ffb1418ea24b1507d229f3c4/users/360d89d785164e0f9b8be6795339482a/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:48.462195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dfb8c-a53b-43b0-95e2-97f8351d0cf5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:48.568017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8862565d7ca44216b7dc76e8df01f1cd', 'request_id': 'req-a8615328-e039-41e9-859d-a63ca314e61d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:48.582200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8862565d7ca44216b7dc76e8df01f1cd', 'request_id': 'req-a8615328-e039-41e9-859d-a63ca314e61d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:48.590618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8862565d7ca44216b7dc76e8df01f1cd', 'request_id': 'req-a8615328-e039-41e9-859d-a63ca314e61d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:48.618351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] The token cache is being invalidate because project 8862565d7ca44216b7dc76e8df01f1cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:48.618901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:48.620903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a8615328-e039-41e9-859d-a63ca314e61d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:48.630530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1044/2107] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:48 2022] DELETE /identity/v3/projects/8862565d7ca44216b7dc76e8df01f1cd => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:48.631865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61251b3-bcfe-43a5-b7a6-a4f4fea7499a None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:48.632155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61251b3-bcfe-43a5-b7a6-a4f4fea7499a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:48.633059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61251b3-bcfe-43a5-b7a6-a4f4fea7499a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:48.633819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dfb8c-a53b-43b0-95e2-97f8351d0cf5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:48.666201 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-c61251b3-bcfe-43a5-b7a6-a4f4fea7499a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:48.667643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c61251b3-bcfe-43a5-b7a6-a4f4fea7499a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:48.768825 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:48.769532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5bda9-53ef-4218-aade-8b30e77fab85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:48.770227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5bda9-53ef-4218-aade-8b30e77fab85 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:48.771262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5bda9-53ef-4218-aade-8b30e77fab85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:48.771898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5bda9-53ef-4218-aade-8b30e77fab85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:48.774347 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5bda9-53ef-4218-aade-8b30e77fab85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:48.779301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5bda9-53ef-4218-aade-8b30e77fab85 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:48.807302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1064/2108] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:48.811526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5bda9-53ef-4218-aade-8b30e77fab85 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:48.818059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7fe5bda9-53ef-4218-aade-8b30e77fab85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:48.869486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5bda9-53ef-4218-aade-8b30e77fab85 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:48.878821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1045/2109] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:48.881347 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00019be4-0e2b-4c29-a0f9-a410f2b33d9f None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:48.881650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00019be4-0e2b-4c29-a0f9-a410f2b33d9f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:48.881930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00019be4-0e2b-4c29-a0f9-a410f2b33d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:48.885366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b37c1-7192-4149-86fc-cc216ec0b4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:48.886326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b37c1-7192-4149-86fc-cc216ec0b4af admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:48.886637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b37c1-7192-4149-86fc-cc216ec0b4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:48.886958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b37c1-7192-4149-86fc-cc216ec0b4af admin admin] PATH_INFO: `/v3/projects/9d0e7518ffb1418ea24b1507d229f3c4/users/360d89d785164e0f9b8be6795339482a/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:48.892591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b37c1-7192-4149-86fc-cc216ec0b4af admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0e7518ffb1418ea24b1507d229f3c4, user_id=360d89d785164e0f9b8be6795339482a, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:48.903796 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-00019be4-0e2b-4c29-a0f9-a410f2b33d9f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:48.905032 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-00019be4-0e2b-4c29-a0f9-a410f2b33d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:48.905941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b37c1-7192-4149-86fc-cc216ec0b4af admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:48.921341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1046/2111] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:48.921341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00019be4-0e2b-4c29-a0f9-a410f2b33d9f None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:48.921812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1065/2111] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:48 2022] PUT /identity/v3/projects/9d0e7518ffb1418ea24b1507d229f3c4/users/360d89d785164e0f9b8be6795339482a/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:48.930711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00019be4-0e2b-4c29-a0f9-a410f2b33d9f None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:49.022026 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299552c-24b3-4221-b8bc-3d1762b9624e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:49.022753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299552c-24b3-4221-b8bc-3d1762b9624e admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:49.023047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299552c-24b3-4221-b8bc-3d1762b9624e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:49.023325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299552c-24b3-4221-b8bc-3d1762b9624e admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:49.024072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299552c-24b3-4221-b8bc-3d1762b9624e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:49.024786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299552c-24b3-4221-b8bc-3d1762b9624e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:49.026438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299552c-24b3-4221-b8bc-3d1762b9624e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:49.044828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1047/2112] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:49.048399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b37c1-7192-4149-86fc-cc216ec0b4af admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:49.067851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b37c1-7192-4149-86fc-cc216ec0b4af admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:49.073561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d8e2c-884d-4dae-b8c5-ebbcd029381f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:49.074508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d8e2c-884d-4dae-b8c5-ebbcd029381f admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:49.074981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d8e2c-884d-4dae-b8c5-ebbcd029381f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:49.075402 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d8e2c-884d-4dae-b8c5-ebbcd029381f admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:49.076305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d8e2c-884d-4dae-b8c5-ebbcd029381f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:49.078446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d8e2c-884d-4dae-b8c5-ebbcd029381f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:49.112653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1066/2113] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:49 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:49.116875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299552c-24b3-4221-b8bc-3d1762b9624e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:49.129428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299552c-24b3-4221-b8bc-3d1762b9624e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:49.249388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707d28d-e501-4c12-9913-629e6c421986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:49.250329 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d28d-e501-4c12-9913-629e6c421986 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:49.251372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d28d-e501-4c12-9913-629e6c421986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:49.251876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d28d-e501-4c12-9913-629e6c421986 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:49.252995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707d28d-e501-4c12-9913-629e6c421986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:49.255336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707d28d-e501-4c12-9913-629e6c421986 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:49.285636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1048/2114] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:49 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:49.289405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d8e2c-884d-4dae-b8c5-ebbcd029381f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:49.306706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d8e2c-884d-4dae-b8c5-ebbcd029381f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:49.328044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7531fdb-890e-4aa7-9b56-bf436ff09d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:49.328762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531fdb-890e-4aa7-9b56-bf436ff09d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:49.329046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531fdb-890e-4aa7-9b56-bf436ff09d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:49.329368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531fdb-890e-4aa7-9b56-bf436ff09d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:49.334621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7531fdb-890e-4aa7-9b56-bf436ff09d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:49.336310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7531fdb-890e-4aa7-9b56-bf436ff09d58 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:49.353841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1067/2115] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:49.354750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707d28d-e501-4c12-9913-629e6c421986 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:49.371333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707d28d-e501-4c12-9913-629e6c421986 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:49.388396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402956a8-83ca-4cb6-b4e1-187f63b4a057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:49.389143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402956a8-83ca-4cb6-b4e1-187f63b4a057 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:49.389419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402956a8-83ca-4cb6-b4e1-187f63b4a057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:49.389735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402956a8-83ca-4cb6-b4e1-187f63b4a057 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:49.390665 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402956a8-83ca-4cb6-b4e1-187f63b4a057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:49.393599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402956a8-83ca-4cb6-b4e1-187f63b4a057 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:49.413485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1049/2116] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 09:59:49.415034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7531fdb-890e-4aa7-9b56-bf436ff09d58 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:49.427314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7531fdb-890e-4aa7-9b56-bf436ff09d58 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:49.432515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50287e1b-b36f-4d41-a357-b595706daca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:49.433578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50287e1b-b36f-4d41-a357-b595706daca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:49.433966 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50287e1b-b36f-4d41-a357-b595706daca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:49.434742 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50287e1b-b36f-4d41-a357-b595706daca8 admin admin] PATH_INFO: `/v3/projects/624a95a45e0a4deab02ddc7bed25eebb/users/adc68245dea64db09248d0a66d17e5f4/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:49.458101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:49.467262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50287e1b-b36f-4d41-a357-b595706daca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624a95a45e0a4deab02ddc7bed25eebb, user_id=adc68245dea64db09248d0a66d17e5f4, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:49.473216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50287e1b-b36f-4d41-a357-b595706daca8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:49.511969 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1068/2117] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:49 2022] PUT /identity/v3/projects/624a95a45e0a4deab02ddc7bed25eebb/users/adc68245dea64db09248d0a66d17e5f4/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:49.512319 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad2cd4-9055-4c1c-8bfc-2cbaa618ddcb None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:49.512590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad2cd4-9055-4c1c-8bfc-2cbaa618ddcb None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:49.512865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad2cd4-9055-4c1c-8bfc-2cbaa618ddcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:49.553660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-c8ad2cd4-9055-4c1c-8bfc-2cbaa618ddcb None None] MFA Rules not processed for user `adc68245dea64db09248d0a66d17e5f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:49.554602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c8ad2cd4-9055-4c1c-8bfc-2cbaa618ddcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:49.613775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:49.669060 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1050/2118] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 09:59:49.678918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50287e1b-b36f-4d41-a357-b595706daca8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:49.681200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-50287e1b-b36f-4d41-a357-b595706daca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:49.786697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50287e1b-b36f-4d41-a357-b595706daca8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:49.793012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e6a04-3c71-40dd-8282-a71fbe4109a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:49.794218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e6a04-3c71-40dd-8282-a71fbe4109a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:49.794669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e6a04-3c71-40dd-8282-a71fbe4109a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:49.795130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e6a04-3c71-40dd-8282-a71fbe4109a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:49.826333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e6a04-3c71-40dd-8282-a71fbe4109a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:49.831054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e6a04-3c71-40dd-8282-a71fbe4109a5 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:49.861872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1069/2119] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:51.073925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03ca0b-c9f7-42d0-80e9-c64c13486f48 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:51.074533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03ca0b-c9f7-42d0-80e9-c64c13486f48 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:51.074533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03ca0b-c9f7-42d0-80e9-c64c13486f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:51.119586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fa03ca0b-c9f7-42d0-80e9-c64c13486f48 None None] MFA Rules not processed for user `1694bd50d03f4d0f9c2ccee1b0775e12`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:51.120895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fa03ca0b-c9f7-42d0-80e9-c64c13486f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:51.232280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1051/2120] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:51.751712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548e15f-d46d-4ded-9e80-6aa29e3a17f3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:51.752231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548e15f-d46d-4ded-9e80-6aa29e3a17f3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:51.752440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548e15f-d46d-4ded-9e80-6aa29e3a17f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:51.791027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e548e15f-d46d-4ded-9e80-6aa29e3a17f3 None None] MFA Rules not processed for user `360d89d785164e0f9b8be6795339482a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:51.792151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e548e15f-d46d-4ded-9e80-6aa29e3a17f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:51.938109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1070/2121] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:51.956285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03ca0b-c9f7-42d0-80e9-c64c13486f48 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:51.976312 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03ca0b-c9f7-42d0-80e9-c64c13486f48 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:51.993561 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af49e66-708d-4624-8835-661a3232474a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:52.001153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af49e66-708d-4624-8835-661a3232474a service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:52.001449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af49e66-708d-4624-8835-661a3232474a service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:52.001725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af49e66-708d-4624-8835-661a3232474a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:52.006989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8af49e66-708d-4624-8835-661a3232474a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:52.247537 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af49e66-708d-4624-8835-661a3232474a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:52.249063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af49e66-708d-4624-8835-661a3232474a service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:52.305090 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1052/2122] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 09:59:54.149720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548e15f-d46d-4ded-9e80-6aa29e3a17f3 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:54.151308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e548e15f-d46d-4ded-9e80-6aa29e3a17f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:54.190973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548e15f-d46d-4ded-9e80-6aa29e3a17f3 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:54.235941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097f250-a3cd-4377-adf4-f918a5f86299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:54.237876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097f250-a3cd-4377-adf4-f918a5f86299 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:54.238167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097f250-a3cd-4377-adf4-f918a5f86299 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:54.239773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097f250-a3cd-4377-adf4-f918a5f86299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:54.282759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097f250-a3cd-4377-adf4-f918a5f86299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:54.288062 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097f250-a3cd-4377-adf4-f918a5f86299 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:54.314459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1071/2123] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:55.104879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7c3e2-1d4e-4eee-b150-71ad016a2988 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:55.107158 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7c3e2-1d4e-4eee-b150-71ad016a2988 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:55.107660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7c3e2-1d4e-4eee-b150-71ad016a2988 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:55.110859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1053/2124] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 09:59:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 09:59:55.121262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077d3f2-1e87-4422-8fab-be966848f7dc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:55.121457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077d3f2-1e87-4422-8fab-be966848f7dc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:55.121956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077d3f2-1e87-4422-8fab-be966848f7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:55.164216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1077d3f2-1e87-4422-8fab-be966848f7dc None None] MFA Rules not processed for user `e9bff6c855ea46feaa5526d2c27c9026`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:55.166916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1077d3f2-1e87-4422-8fab-be966848f7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:55.284017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4e7fd-73d8-4c46-912a-916cfbd562d7 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:55.284846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4e7fd-73d8-4c46-912a-916cfbd562d7 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:55.284846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4e7fd-73d8-4c46-912a-916cfbd562d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:55.336148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-21b4e7fd-73d8-4c46-912a-916cfbd562d7 None None] MFA Rules not processed for user `5a2d88a23143416083a691af22d70a66`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:55.337994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-21b4e7fd-73d8-4c46-912a-916cfbd562d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:55.377743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1072/2125] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 09:59:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:55.380623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077d3f2-1e87-4422-8fab-be966848f7dc None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:55.384537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1077d3f2-1e87-4422-8fab-be966848f7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:55.392192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:55.518386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077d3f2-1e87-4422-8fab-be966848f7dc None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:55.525108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:55.526293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:55.527091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:55.528223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] PATH_INFO: `/v3/users/57605be4d856407f9ac4828ee1fef041` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:55.529818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57605be4d856407f9ac4828ee1fef041)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:55.532425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:55.589114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1054/2126] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:55.603948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4e7fd-73d8-4c46-912a-916cfbd562d7 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:55.606778 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-21b4e7fd-73d8-4c46-912a-916cfbd562d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:55.640710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:55.642149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57605be4d856407f9ac4828ee1fef041'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:55.671683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57605be4d856407f9ac4828ee1fef041'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:55.672720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57605be4d856407f9ac4828ee1fef041'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:55.680949 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4e7fd-73d8-4c46-912a-916cfbd562d7 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:55.690503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57605be4d856407f9ac4828ee1fef041'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:55.692221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57605be4d856407f9ac4828ee1fef041'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:55.701868 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824e2eb-67ce-431f-9e0d-ce3e51be94d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:55.702941 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824e2eb-67ce-431f-9e0d-ce3e51be94d0 service placement] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:55.703432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824e2eb-67ce-431f-9e0d-ce3e51be94d0 service placement] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:55.704864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824e2eb-67ce-431f-9e0d-ce3e51be94d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:55.712561 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b824e2eb-67ce-431f-9e0d-ce3e51be94d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:55.725901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1073/2127] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 09:59:55 2022] DELETE /identity/v3/users/57605be4d856407f9ac4828ee1fef041 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:55.727593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:55.731090 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:55.747497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824e2eb-67ce-431f-9e0d-ce3e51be94d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:55.752714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824e2eb-67ce-431f-9e0d-ce3e51be94d0 service placement] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:55.773318 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1055/2128] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:55.939436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472f946-17ba-4b52-8387-9f5719406aa8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:55.951217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380fe9d-e39c-4b3f-bb48-223b6da96e86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:55.952067 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380fe9d-e39c-4b3f-bb48-223b6da96e86 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:55.952361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380fe9d-e39c-4b3f-bb48-223b6da96e86 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:55.952774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380fe9d-e39c-4b3f-bb48-223b6da96e86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:55.955791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5380fe9d-e39c-4b3f-bb48-223b6da96e86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:55.994853 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:56.076178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380fe9d-e39c-4b3f-bb48-223b6da96e86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:56.079198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380fe9d-e39c-4b3f-bb48-223b6da96e86 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:56.101588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1074/2129] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 09:59:56.451554 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824e2eb-67ce-431f-9e0d-ce3e51be94d0 service placement] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:56.453097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b824e2eb-67ce-431f-9e0d-ce3e51be94d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:56.484338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824e2eb-67ce-431f-9e0d-ce3e51be94d0 service placement] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:56.640771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:56.642932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:56.643626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:56.644208 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] PATH_INFO: `/v3/projects/e10f49f1727f4327be986e4f7e131200` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:56.645737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e10f49f1727f4327be986e4f7e131200)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:56.648183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:56.733662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e10f49f1727f4327be986e4f7e131200', 'request_id': 'req-7b776cf8-0fff-452e-8ad8-b0014de4c54b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:56.745612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e10f49f1727f4327be986e4f7e131200', 'request_id': 'req-7b776cf8-0fff-452e-8ad8-b0014de4c54b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:56.756546 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e10f49f1727f4327be986e4f7e131200', 'request_id': 'req-7b776cf8-0fff-452e-8ad8-b0014de4c54b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:56.772480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] The token cache is being invalidate because project e10f49f1727f4327be986e4f7e131200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 09:59:56.772784 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:56.773517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7b776cf8-0fff-452e-8ad8-b0014de4c54b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 09:59:56.775590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1056/2130] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 09:59:56 2022] DELETE /identity/v3/projects/e10f49f1727f4327be986e4f7e131200 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:56.781298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b832d0-b108-438f-a069-98c6131e4685 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:56.781872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b832d0-b108-438f-a069-98c6131e4685 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:56.782299 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b832d0-b108-438f-a069-98c6131e4685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:56.811778 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e5b832d0-b108-438f-a069-98c6131e4685 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:56.814002 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e5b832d0-b108-438f-a069-98c6131e4685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:56.900905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1075/2131] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:56.910280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae57bdf-2df1-4138-bca1-de8eda21b702 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:56.910859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae57bdf-2df1-4138-bca1-de8eda21b702 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:56.911224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae57bdf-2df1-4138-bca1-de8eda21b702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:56.933844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-8ae57bdf-2df1-4138-bca1-de8eda21b702 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:56.935167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8ae57bdf-2df1-4138-bca1-de8eda21b702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:56.948339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1057/2132] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:56.949498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b832d0-b108-438f-a069-98c6131e4685 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:56.962782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b832d0-b108-438f-a069-98c6131e4685 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:56.975210 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34f5aa-f84b-47ab-ba95-3db6bc258f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:56.976050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34f5aa-f84b-47ab-ba95-3db6bc258f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:56.976398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34f5aa-f84b-47ab-ba95-3db6bc258f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:56.976744 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34f5aa-f84b-47ab-ba95-3db6bc258f01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:56.977655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34f5aa-f84b-47ab-ba95-3db6bc258f01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 09:59:56.978026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34f5aa-f84b-47ab-ba95-3db6bc258f01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:56.983910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34f5aa-f84b-47ab-ba95-3db6bc258f01 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:57.015838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1076/2133] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 09:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 09:59:57.017035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae57bdf-2df1-4138-bca1-de8eda21b702 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:57.038317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae57bdf-2df1-4138-bca1-de8eda21b702 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:57.071556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ff31c-8caa-4560-beef-c91b01afd8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:57.072623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ff31c-8caa-4560-beef-c91b01afd8a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:57.072969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ff31c-8caa-4560-beef-c91b01afd8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:57.073311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ff31c-8caa-4560-beef-c91b01afd8a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:57.074137 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ff31c-8caa-4560-beef-c91b01afd8a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:57.076316 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ff31c-8caa-4560-beef-c91b01afd8a9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:57.138978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34f5aa-f84b-47ab-ba95-3db6bc258f01 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:57.139601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1058/2134] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 09:59:57 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 09:59:57.160337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34f5aa-f84b-47ab-ba95-3db6bc258f01 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:57.398663 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff0011-ab0c-495a-a206-c43246aec79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:57.399391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0011-ab0c-495a-a206-c43246aec79f admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:57.399674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0011-ab0c-495a-a206-c43246aec79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:57.399951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0011-ab0c-495a-a206-c43246aec79f admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:57.400659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff0011-ab0c-495a-a206-c43246aec79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:57.410958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff0011-ab0c-495a-a206-c43246aec79f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:57.464706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1077/2135] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 09:59:57 2022] POST /identity/v3/users => generated 293 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 09:59:57.466240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ff31c-8caa-4560-beef-c91b01afd8a9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:57.486604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ff31c-8caa-4560-beef-c91b01afd8a9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:57.499535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e20457-e3fa-4fd9-bb03-73deab831ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:57.502299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e20457-e3fa-4fd9-bb03-73deab831ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:57.502737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e20457-e3fa-4fd9-bb03-73deab831ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:57.504540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e20457-e3fa-4fd9-bb03-73deab831ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:57.508268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e20457-e3fa-4fd9-bb03-73deab831ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:57.510795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e20457-e3fa-4fd9-bb03-73deab831ba1 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:57.542701 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1059/2136] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:57.544547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff0011-ab0c-495a-a206-c43246aec79f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:57.565798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff0011-ab0c-495a-a206-c43246aec79f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:57.592059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afef21-86f9-4079-a0cb-cfe8524d990b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:57.593932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afef21-86f9-4079-a0cb-cfe8524d990b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:57.594307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afef21-86f9-4079-a0cb-cfe8524d990b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:57.594727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afef21-86f9-4079-a0cb-cfe8524d990b admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:57.596217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afef21-86f9-4079-a0cb-cfe8524d990b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:57.601150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afef21-86f9-4079-a0cb-cfe8524d990b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:57.641370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1078/2137] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 09:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 09:59:57.645173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e20457-e3fa-4fd9-bb03-73deab831ba1 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:57.660288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e20457-e3fa-4fd9-bb03-73deab831ba1 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:57.675516 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40849573-193d-4767-a2f0-d36e392394b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:57.676443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40849573-193d-4767-a2f0-d36e392394b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:57.676726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40849573-193d-4767-a2f0-d36e392394b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:57.677006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40849573-193d-4767-a2f0-d36e392394b1 admin admin] PATH_INFO: `/v3/projects/08d103b7e2db492691cda72117de6142/users/587c22b1c33649119b6c9bfa5f77121a/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:57.708845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40849573-193d-4767-a2f0-d36e392394b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d103b7e2db492691cda72117de6142, user_id=587c22b1c33649119b6c9bfa5f77121a, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:57.716674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40849573-193d-4767-a2f0-d36e392394b1 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:57.753553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1060/2138] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 09:59:57 2022] PUT /identity/v3/projects/08d103b7e2db492691cda72117de6142/users/587c22b1c33649119b6c9bfa5f77121a/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 09:59:57.763663 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd782b-388e-47be-8b3f-85a29fa01f62 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:57.764150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd782b-388e-47be-8b3f-85a29fa01f62 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:57.764526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd782b-388e-47be-8b3f-85a29fa01f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:57.832685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f3cd782b-388e-47be-8b3f-85a29fa01f62 None None] MFA Rules not processed for user `587c22b1c33649119b6c9bfa5f77121a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 09:59:57.833911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f3cd782b-388e-47be-8b3f-85a29fa01f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:57.874539 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:58.022591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1079/2139] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 09:59:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 09:59:58.036507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40849573-193d-4767-a2f0-d36e392394b1 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 09:59:58.038725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-40849573-193d-4767-a2f0-d36e392394b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 09:59:58.087405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 09:59:58.214180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40849573-193d-4767-a2f0-d36e392394b1 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 09:59:58.246094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f15e7-adfd-45d5-90ac-e939daa69eb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 09:59:58.246950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f15e7-adfd-45d5-90ac-e939daa69eb5 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 09:59:58.247342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f15e7-adfd-45d5-90ac-e939daa69eb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 09:59:58.247844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f15e7-adfd-45d5-90ac-e939daa69eb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 09:59:58.262697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f15e7-adfd-45d5-90ac-e939daa69eb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 09:59:58.264773 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f15e7-adfd-45d5-90ac-e939daa69eb5 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 09:59:58.295133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1061/2140] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 09:59:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:05.651058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd782b-388e-47be-8b3f-85a29fa01f62 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:05.652655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f3cd782b-388e-47be-8b3f-85a29fa01f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:05.738108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd782b-388e-47be-8b3f-85a29fa01f62 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:05.743267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:05.744168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:05.744445 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:05.744714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] PATH_INFO: `/v3/users/adc68245dea64db09248d0a66d17e5f4` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:05.745949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] RBAC: Authorizing `identity:delete_user(user_id=adc68245dea64db09248d0a66d17e5f4)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:05.747946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:05.810437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adc68245dea64db09248d0a66d17e5f4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:05.831178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adc68245dea64db09248d0a66d17e5f4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:05.832489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adc68245dea64db09248d0a66d17e5f4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:05.843943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adc68245dea64db09248d0a66d17e5f4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:05.844788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adc68245dea64db09248d0a66d17e5f4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:05.867420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1080/2141] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:05 2022] DELETE /identity/v3/users/adc68245dea64db09248d0a66d17e5f4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:05.873906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d91492-2ba2-41ba-9042-03f13fd43c24 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:05.874444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d91492-2ba2-41ba-9042-03f13fd43c24 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:05.874444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d91492-2ba2-41ba-9042-03f13fd43c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:05.893717 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b6d91492-2ba2-41ba-9042-03f13fd43c24 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:05.895208 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b6d91492-2ba2-41ba-9042-03f13fd43c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:05.963167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1062/2142] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:05.974587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:05.976417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:06.021210 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:06.049081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b98fd-e088-450e-9e54-94b7757db0be admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:06.054005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df190579-2757-483e-a856-8e34c401ef52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:06.055149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df190579-2757-483e-a856-8e34c401ef52 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:06.055597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df190579-2757-483e-a856-8e34c401ef52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:06.056126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df190579-2757-483e-a856-8e34c401ef52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:06.078277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df190579-2757-483e-a856-8e34c401ef52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:06.080064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df190579-2757-483e-a856-8e34c401ef52 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:06.092798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1081/2143] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:06.204934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d91492-2ba2-41ba-9042-03f13fd43c24 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:06.205931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b6d91492-2ba2-41ba-9042-03f13fd43c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:06.225018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d91492-2ba2-41ba-9042-03f13fd43c24 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:06.234066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:06.235149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:06.235505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:06.235854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] PATH_INFO: `/v3/users/587c22b1c33649119b6c9bfa5f77121a` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:06.237246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=587c22b1c33649119b6c9bfa5f77121a)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:06.239539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:06.304228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '587c22b1c33649119b6c9bfa5f77121a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:06.320426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '587c22b1c33649119b6c9bfa5f77121a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:06.321369 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '587c22b1c33649119b6c9bfa5f77121a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:06.322558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '587c22b1c33649119b6c9bfa5f77121a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:06.334229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '587c22b1c33649119b6c9bfa5f77121a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:06.351737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1063/2144] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:06 2022] DELETE /identity/v3/users/587c22b1c33649119b6c9bfa5f77121a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:06.356520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec98a11-7e3d-4940-a87f-f275889d2599 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:06.356803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec98a11-7e3d-4940-a87f-f275889d2599 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:06.357072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec98a11-7e3d-4940-a87f-f275889d2599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:06.382115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-eec98a11-7e3d-4940-a87f-f275889d2599 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:06.383683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-eec98a11-7e3d-4940-a87f-f275889d2599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:06.473089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:06.475009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:06.492077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1082/2145] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:06.494959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545cace-a15d-4195-ac5f-a2cc9d5353fc admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:06.497093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec98a11-7e3d-4940-a87f-f275889d2599 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:06.499139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-eec98a11-7e3d-4940-a87f-f275889d2599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:06.500233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:06.501277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:06.501559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:06.502136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] PATH_INFO: `/v3/projects/624a95a45e0a4deab02ddc7bed25eebb` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:06.503433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=624a95a45e0a4deab02ddc7bed25eebb)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:06.505309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:06.527282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:06.582800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624a95a45e0a4deab02ddc7bed25eebb', 'request_id': 'req-2a4e1964-fbdb-450d-bb48-3b4727a6da94'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:06.588123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec98a11-7e3d-4940-a87f-f275889d2599 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:06.595330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624a95a45e0a4deab02ddc7bed25eebb', 'request_id': 'req-2a4e1964-fbdb-450d-bb48-3b4727a6da94'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:06.605564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624a95a45e0a4deab02ddc7bed25eebb', 'request_id': 'req-2a4e1964-fbdb-450d-bb48-3b4727a6da94'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:06.617653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] The token cache is being invalidate because project 624a95a45e0a4deab02ddc7bed25eebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:06.618022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:06.618780 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:06.620692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1064/2146] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:06 2022] DELETE /identity/v3/projects/624a95a45e0a4deab02ddc7bed25eebb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:00:06.647092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b151ce-7d4f-463c-a994-d7da056354ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:06.647841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b151ce-7d4f-463c-a994-d7da056354ef service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:06.648117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b151ce-7d4f-463c-a994-d7da056354ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:06.648383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b151ce-7d4f-463c-a994-d7da056354ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:06.650679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-63b151ce-7d4f-463c-a994-d7da056354ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:06.704184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:06.726769 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b151ce-7d4f-463c-a994-d7da056354ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:06.728318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b151ce-7d4f-463c-a994-d7da056354ef service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:06.740713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1083/2147] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:00:07.092715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:07.094100 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:07.108256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e1964-fbdb-450d-bb48-3b4727a6da94 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:07.113925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:07.114980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:07.115919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:07.115919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] PATH_INFO: `/v3/projects/08d103b7e2db492691cda72117de6142` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:07.117232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d103b7e2db492691cda72117de6142)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:07.122120 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:07.185955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d103b7e2db492691cda72117de6142', 'request_id': 'req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:07.205051 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d103b7e2db492691cda72117de6142', 'request_id': 'req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:07.229543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d103b7e2db492691cda72117de6142', 'request_id': 'req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:07.246383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] The token cache is being invalidate because project 08d103b7e2db492691cda72117de6142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:07.246712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:07.249547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a413a0c1-8e28-4124-ba6f-57d2ca17b8cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:07.260298 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1065/2148] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:07 2022] DELETE /identity/v3/projects/08d103b7e2db492691cda72117de6142 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:07.264479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810a2c6-8fa7-4963-9592-984d3b4e9a27 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:07.264866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810a2c6-8fa7-4963-9592-984d3b4e9a27 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:07.265139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810a2c6-8fa7-4963-9592-984d3b4e9a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:07.317543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1810a2c6-8fa7-4963-9592-984d3b4e9a27 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:07.322618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1810a2c6-8fa7-4963-9592-984d3b4e9a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:07.397377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1084/2149] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:07.405209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada39a6-8342-4d7a-b65a-48047be69cdf None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:07.405482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada39a6-8342-4d7a-b65a-48047be69cdf None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:07.405776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada39a6-8342-4d7a-b65a-48047be69cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:07.425465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-eada39a6-8342-4d7a-b65a-48047be69cdf None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:07.427081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-eada39a6-8342-4d7a-b65a-48047be69cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:07.436209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1066/2150] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:00:07.440861 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810a2c6-8fa7-4963-9592-984d3b4e9a27 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:07.451496 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810a2c6-8fa7-4963-9592-984d3b4e9a27 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:07.458271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5826506-2233-4d6c-ac4e-97eff3a97c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:07.459183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5826506-2233-4d6c-ac4e-97eff3a97c6e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:07.459487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5826506-2233-4d6c-ac4e-97eff3a97c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:07.463905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5826506-2233-4d6c-ac4e-97eff3a97c6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:07.463905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5826506-2233-4d6c-ac4e-97eff3a97c6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:00:07.463905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5826506-2233-4d6c-ac4e-97eff3a97c6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:07.465824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5826506-2233-4d6c-ac4e-97eff3a97c6e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:07.480318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1085/2151] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:00:07.485052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada39a6-8342-4d7a-b65a-48047be69cdf None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:07.495396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada39a6-8342-4d7a-b65a-48047be69cdf None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:07.501018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426729c-71a5-4a29-8413-d2733f8cd9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:07.501821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426729c-71a5-4a29-8413-d2733f8cd9dd admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:07.502080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426729c-71a5-4a29-8413-d2733f8cd9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:07.502444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426729c-71a5-4a29-8413-d2733f8cd9dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:07.504176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426729c-71a5-4a29-8413-d2733f8cd9dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:07.505883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426729c-71a5-4a29-8413-d2733f8cd9dd admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:07.552641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1067/2152] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:07 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:07.556753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5826506-2233-4d6c-ac4e-97eff3a97c6e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:07.570686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5826506-2233-4d6c-ac4e-97eff3a97c6e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:07.673894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b350a9-24ec-438e-8ecc-799508bdd0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:07.675023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b350a9-24ec-438e-8ecc-799508bdd0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:07.675421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b350a9-24ec-438e-8ecc-799508bdd0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:07.675661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b350a9-24ec-438e-8ecc-799508bdd0c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:07.676584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b350a9-24ec-438e-8ecc-799508bdd0c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:07.678818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b350a9-24ec-438e-8ecc-799508bdd0c0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:07.705236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1086/2153] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:07 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:00:07.709706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426729c-71a5-4a29-8413-d2733f8cd9dd admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:07.722008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426729c-71a5-4a29-8413-d2733f8cd9dd admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:07.727697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65102858-02aa-4cc4-a646-13a89ffb22b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:07.728443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102858-02aa-4cc4-a646-13a89ffb22b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:07.728739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102858-02aa-4cc4-a646-13a89ffb22b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:07.728992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102858-02aa-4cc4-a646-13a89ffb22b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:07.729820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65102858-02aa-4cc4-a646-13a89ffb22b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:07.731577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65102858-02aa-4cc4-a646-13a89ffb22b3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:07.746602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1068/2154] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:07.749457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b350a9-24ec-438e-8ecc-799508bdd0c0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:07.761100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b350a9-24ec-438e-8ecc-799508bdd0c0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:07.767176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7a7aa-8da6-4006-8f61-f82c3b29a16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:07.767874 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a7aa-8da6-4006-8f61-f82c3b29a16d admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:07.768265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a7aa-8da6-4006-8f61-f82c3b29a16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:07.768634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a7aa-8da6-4006-8f61-f82c3b29a16d admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:07.769400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7a7aa-8da6-4006-8f61-f82c3b29a16d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:07.771136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7a7aa-8da6-4006-8f61-f82c3b29a16d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:07.786474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1087/2155] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:07.787715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65102858-02aa-4cc4-a646-13a89ffb22b3 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:07.798842 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65102858-02aa-4cc4-a646-13a89ffb22b3 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:07.803759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee19e9b-44fc-48ed-bdfe-0e4eded05eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:07.804610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee19e9b-44fc-48ed-bdfe-0e4eded05eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:07.804916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee19e9b-44fc-48ed-bdfe-0e4eded05eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:07.805192 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee19e9b-44fc-48ed-bdfe-0e4eded05eb9 admin admin] PATH_INFO: `/v3/projects/95b3aa9725844803a5fd70bd90e678a6/users/4e774d2a8a2f48f892900bd263222081/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:07.823236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee19e9b-44fc-48ed-bdfe-0e4eded05eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b3aa9725844803a5fd70bd90e678a6, user_id=4e774d2a8a2f48f892900bd263222081, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:07.826125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee19e9b-44fc-48ed-bdfe-0e4eded05eb9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:07.842850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1069/2156] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:07 2022] PUT /identity/v3/projects/95b3aa9725844803a5fd70bd90e678a6/users/4e774d2a8a2f48f892900bd263222081/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:07.858225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6729ae2-8e0b-4ad9-a3b8-483662be8175 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:07.858555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6729ae2-8e0b-4ad9-a3b8-483662be8175 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:07.859064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6729ae2-8e0b-4ad9-a3b8-483662be8175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:07.898459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d6729ae2-8e0b-4ad9-a3b8-483662be8175 None None] MFA Rules not processed for user `4e774d2a8a2f48f892900bd263222081`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:07.900121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d6729ae2-8e0b-4ad9-a3b8-483662be8175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:07.962971 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:07.999896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1088/2157] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:08.005015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee19e9b-44fc-48ed-bdfe-0e4eded05eb9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:08.027019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:08.086169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee19e9b-44fc-48ed-bdfe-0e4eded05eb9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:08.091586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbf4d3-2e63-48d6-b686-898827dce4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:08.092233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbf4d3-2e63-48d6-b686-898827dce4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:08.092519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbf4d3-2e63-48d6-b686-898827dce4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:08.092786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbf4d3-2e63-48d6-b686-898827dce4e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:08.093472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbf4d3-2e63-48d6-b686-898827dce4e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:08.095445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbf4d3-2e63-48d6-b686-898827dce4e9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:08.126061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1070/2158] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:08 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:08.127259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6729ae2-8e0b-4ad9-a3b8-483662be8175 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:08.138927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6729ae2-8e0b-4ad9-a3b8-483662be8175 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:08.200171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c80dc-732f-42c6-8b3b-2bfd9bb287f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:08.200865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c80dc-732f-42c6-8b3b-2bfd9bb287f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:08.201143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c80dc-732f-42c6-8b3b-2bfd9bb287f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:08.201419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c80dc-732f-42c6-8b3b-2bfd9bb287f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:08.202104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c80dc-732f-42c6-8b3b-2bfd9bb287f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:08.204071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c80dc-732f-42c6-8b3b-2bfd9bb287f3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:08.226084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1089/2159] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:08 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:00:08.230288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbf4d3-2e63-48d6-b686-898827dce4e9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:08.241233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbf4d3-2e63-48d6-b686-898827dce4e9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:08.245975 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6ebe8-f30c-42a5-84ab-0a698c245ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:08.246716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6ebe8-f30c-42a5-84ab-0a698c245ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:08.247001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6ebe8-f30c-42a5-84ab-0a698c245ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:08.247281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6ebe8-f30c-42a5-84ab-0a698c245ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:08.248024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6ebe8-f30c-42a5-84ab-0a698c245ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:08.249667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6ebe8-f30c-42a5-84ab-0a698c245ab3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:08.261194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1071/2160] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:08.265982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c80dc-732f-42c6-8b3b-2bfd9bb287f3 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:08.277040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c80dc-732f-42c6-8b3b-2bfd9bb287f3 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:08.283103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a87000-7f5c-4502-a02f-d7a76077f3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:08.284667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a87000-7f5c-4502-a02f-d7a76077f3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:08.284667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a87000-7f5c-4502-a02f-d7a76077f3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:08.284872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a87000-7f5c-4502-a02f-d7a76077f3d5 admin admin] PATH_INFO: `/v3/projects/d5302df983854dea82f795b0a58a67d5/users/76fb2c5774484fd79e6afd670aa5c19a/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:08.303862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a87000-7f5c-4502-a02f-d7a76077f3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5302df983854dea82f795b0a58a67d5, user_id=76fb2c5774484fd79e6afd670aa5c19a, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:08.307176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a87000-7f5c-4502-a02f-d7a76077f3d5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:08.323330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1090/2161] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:08 2022] PUT /identity/v3/projects/d5302df983854dea82f795b0a58a67d5/users/76fb2c5774484fd79e6afd670aa5c19a/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:08.327592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6ebe8-f30c-42a5-84ab-0a698c245ab3 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:08.340232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6ebe8-f30c-42a5-84ab-0a698c245ab3 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:08.417408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980fe80-a313-4b26-b82b-59ba322e13ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:08.418097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980fe80-a313-4b26-b82b-59ba322e13ab admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:08.418379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980fe80-a313-4b26-b82b-59ba322e13ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:08.418689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980fe80-a313-4b26-b82b-59ba322e13ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:08.419626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980fe80-a313-4b26-b82b-59ba322e13ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:08.422055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980fe80-a313-4b26-b82b-59ba322e13ab admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:08.438196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1072/2162] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:08.442389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a87000-7f5c-4502-a02f-d7a76077f3d5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:08.454521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a87000-7f5c-4502-a02f-d7a76077f3d5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:08.460143 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a76d93-70bd-44db-bc09-0ab7e387b527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:08.462390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a76d93-70bd-44db-bc09-0ab7e387b527 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:08.462906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a76d93-70bd-44db-bc09-0ab7e387b527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:08.463383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a76d93-70bd-44db-bc09-0ab7e387b527 admin admin] PATH_INFO: `/v3/projects/d5302df983854dea82f795b0a58a67d5/users/76fb2c5774484fd79e6afd670aa5c19a/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:08.465653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a76d93-70bd-44db-bc09-0ab7e387b527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5302df983854dea82f795b0a58a67d5, user_id=76fb2c5774484fd79e6afd670aa5c19a, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:08.468664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a76d93-70bd-44db-bc09-0ab7e387b527 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:08.481307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1091/2163] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:08 2022] PUT /identity/v3/projects/d5302df983854dea82f795b0a58a67d5/users/76fb2c5774484fd79e6afd670aa5c19a/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:00:08.492864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e229453-f2a7-4661-9fb9-17225ce093b5 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:08.493178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e229453-f2a7-4661-9fb9-17225ce093b5 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:08.493568 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e229453-f2a7-4661-9fb9-17225ce093b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:08.539781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2e229453-f2a7-4661-9fb9-17225ce093b5 None None] MFA Rules not processed for user `76fb2c5774484fd79e6afd670aa5c19a`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:08.541898 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2e229453-f2a7-4661-9fb9-17225ce093b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:08.663035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1073/2164] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:00:08.739496 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a76d93-70bd-44db-bc09-0ab7e387b527 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:08.741339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f3a76d93-70bd-44db-bc09-0ab7e387b527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:08.829376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a76d93-70bd-44db-bc09-0ab7e387b527 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:08.834772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92feaaa8-0c56-4628-bc19-a9b86c6065f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:08.835661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92feaaa8-0c56-4628-bc19-a9b86c6065f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:08.835856 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92feaaa8-0c56-4628-bc19-a9b86c6065f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:08.836718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92feaaa8-0c56-4628-bc19-a9b86c6065f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:08.848733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92feaaa8-0c56-4628-bc19-a9b86c6065f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:08.850250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92feaaa8-0c56-4628-bc19-a9b86c6065f2 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:08.861101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1092/2165] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:09.467738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e229453-f2a7-4661-9fb9-17225ce093b5 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:09.485493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:09.540079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e229453-f2a7-4661-9fb9-17225ce093b5 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:09.545159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:09.546159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:09.546445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:09.546753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] PATH_INFO: `/v3/users/4e774d2a8a2f48f892900bd263222081` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:09.547992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e774d2a8a2f48f892900bd263222081)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:09.550252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:09.605884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e774d2a8a2f48f892900bd263222081'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:09.620550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e774d2a8a2f48f892900bd263222081'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:09.621305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e774d2a8a2f48f892900bd263222081'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:09.622059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e774d2a8a2f48f892900bd263222081'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:09.629174 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e774d2a8a2f48f892900bd263222081'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:09.641742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1074/2166] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:09 2022] DELETE /identity/v3/users/4e774d2a8a2f48f892900bd263222081 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:09.646537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f51949-17e2-40ea-aed9-27fc67cc1964 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:09.646860 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f51949-17e2-40ea-aed9-27fc67cc1964 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:09.647134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f51949-17e2-40ea-aed9-27fc67cc1964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:09.666802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-26f51949-17e2-40ea-aed9-27fc67cc1964 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:09.668029 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-26f51949-17e2-40ea-aed9-27fc67cc1964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:09.702020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:09.738083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1093/2167] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:09.747947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:09.749526 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:09.807039 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fcaa4-52eb-4fec-b2d4-35329df9321b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:09.810816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061128-b786-4838-a678-f282cacd5738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:09.811473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061128-b786-4838-a678-f282cacd5738 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:09.811750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061128-b786-4838-a678-f282cacd5738 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:09.812019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061128-b786-4838-a678-f282cacd5738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:09.822838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70061128-b786-4838-a678-f282cacd5738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:09.824302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70061128-b786-4838-a678-f282cacd5738 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:09.837546 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1075/2168] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:10.182471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f51949-17e2-40ea-aed9-27fc67cc1964 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:10.183998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-26f51949-17e2-40ea-aed9-27fc67cc1964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:10.196613 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f51949-17e2-40ea-aed9-27fc67cc1964 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:10.201422 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:10.202472 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:10.202633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:10.202971 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] PATH_INFO: `/v3/projects/95b3aa9725844803a5fd70bd90e678a6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:10.204167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95b3aa9725844803a5fd70bd90e678a6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:10.206479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:10.259804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95b3aa9725844803a5fd70bd90e678a6', 'request_id': 'req-7dff4beb-e586-480e-9709-d475f3dd3f85'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:10.267822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95b3aa9725844803a5fd70bd90e678a6', 'request_id': 'req-7dff4beb-e586-480e-9709-d475f3dd3f85'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:10.275688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95b3aa9725844803a5fd70bd90e678a6', 'request_id': 'req-7dff4beb-e586-480e-9709-d475f3dd3f85'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:10.288578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] The token cache is being invalidate because project 95b3aa9725844803a5fd70bd90e678a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:10.288862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:10.289512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dff4beb-e586-480e-9709-d475f3dd3f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:10.291213 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1094/2169] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:10 2022] DELETE /identity/v3/projects/95b3aa9725844803a5fd70bd90e678a6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:10.295858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061128-b786-4838-a678-f282cacd5738 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:10.308080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061128-b786-4838-a678-f282cacd5738 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:10.348210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:10.377391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92dedf-f8eb-406e-9341-daf4d39ba9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:10.378383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92dedf-f8eb-406e-9341-daf4d39ba9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:10.378683 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92dedf-f8eb-406e-9341-daf4d39ba9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:10.378953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92dedf-f8eb-406e-9341-daf4d39ba9a4 admin admin] PATH_INFO: `/v3/users/76fb2c5774484fd79e6afd670aa5c19a` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:10.380290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92dedf-f8eb-406e-9341-daf4d39ba9a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76fb2c5774484fd79e6afd670aa5c19a)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:10.382260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92dedf-f8eb-406e-9341-daf4d39ba9a4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:10.444920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-eb92dedf-f8eb-406e-9341-daf4d39ba9a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76fb2c5774484fd79e6afd670aa5c19a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:10.468328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-eb92dedf-f8eb-406e-9341-daf4d39ba9a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76fb2c5774484fd79e6afd670aa5c19a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:10.469804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-eb92dedf-f8eb-406e-9341-daf4d39ba9a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76fb2c5774484fd79e6afd670aa5c19a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:10.470620 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-eb92dedf-f8eb-406e-9341-daf4d39ba9a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76fb2c5774484fd79e6afd670aa5c19a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:10.480806 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-eb92dedf-f8eb-406e-9341-daf4d39ba9a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76fb2c5774484fd79e6afd670aa5c19a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:10.501899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1076/2170] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:10 2022] DELETE /identity/v3/users/76fb2c5774484fd79e6afd670aa5c19a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:11.206734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01601f2-e717-43ea-ab14-a7ea38d960c8 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:11.207565 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01601f2-e717-43ea-ab14-a7ea38d960c8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:11.208095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01601f2-e717-43ea-ab14-a7ea38d960c8 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:11.211051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1095/2171] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 10:00:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 10:00:11.223387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251fc32-617b-4ee4-94fd-314429e2b6bf None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:11.223706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251fc32-617b-4ee4-94fd-314429e2b6bf None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:11.226078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251fc32-617b-4ee4-94fd-314429e2b6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:11.278287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-f251fc32-617b-4ee4-94fd-314429e2b6bf None None] MFA Rules not processed for user `e9bff6c855ea46feaa5526d2c27c9026`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:11.283028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f251fc32-617b-4ee4-94fd-314429e2b6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:11.419916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1077/2172] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 10:00:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:00:11.662469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01601f2-e717-43ea-ab14-a7ea38d960c8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:11.666717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e01601f2-e717-43ea-ab14-a7ea38d960c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:11.703806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01601f2-e717-43ea-ab14-a7ea38d960c8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:11.709070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96638a18-c10e-4900-82c9-a4fb9f2d4947 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:11.710999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96638a18-c10e-4900-82c9-a4fb9f2d4947 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:11.712089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96638a18-c10e-4900-82c9-a4fb9f2d4947 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:11.712561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96638a18-c10e-4900-82c9-a4fb9f2d4947 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:11.723491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96638a18-c10e-4900-82c9-a4fb9f2d4947 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:11.725440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96638a18-c10e-4900-82c9-a4fb9f2d4947 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:11.737790 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1096/2173] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:11.790017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251fc32-617b-4ee4-94fd-314429e2b6bf None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:11.791629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f251fc32-617b-4ee4-94fd-314429e2b6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:11.897083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:11.920757 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251fc32-617b-4ee4-94fd-314429e2b6bf None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:11.928873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:11.932948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:11.933470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:11.933927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] PATH_INFO: `/v3/projects/d5302df983854dea82f795b0a58a67d5` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:11.936459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5302df983854dea82f795b0a58a67d5)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:11.939295 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:12.008952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5302df983854dea82f795b0a58a67d5', 'request_id': 'req-57e981cf-69ee-4481-bbb9-2820a1e9dd91'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:12.019676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5302df983854dea82f795b0a58a67d5', 'request_id': 'req-57e981cf-69ee-4481-bbb9-2820a1e9dd91'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:12.028292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5302df983854dea82f795b0a58a67d5', 'request_id': 'req-57e981cf-69ee-4481-bbb9-2820a1e9dd91'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:12.042278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] The token cache is being invalidate because project d5302df983854dea82f795b0a58a67d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:12.042661 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:12.043378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-57e981cf-69ee-4481-bbb9-2820a1e9dd91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:12.045386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1078/2174] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:11 2022] DELETE /identity/v3/projects/d5302df983854dea82f795b0a58a67d5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:12.052431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ae92f-b823-4d15-847d-a594214e21f7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:12.052713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ae92f-b823-4d15-847d-a594214e21f7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:12.052989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ae92f-b823-4d15-847d-a594214e21f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:12.072755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-519ae92f-b823-4d15-847d-a594214e21f7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:12.073954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-519ae92f-b823-4d15-847d-a594214e21f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:12.143082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1097/2175] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:12.151258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d614df-292f-4968-8ec3-185b6dccb1c7 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:12.151542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d614df-292f-4968-8ec3-185b6dccb1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:12.151952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d614df-292f-4968-8ec3-185b6dccb1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:12.171672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-28d614df-292f-4968-8ec3-185b6dccb1c7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:12.172892 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-28d614df-292f-4968-8ec3-185b6dccb1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:12.181371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1079/2176] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:00:12.186285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ae92f-b823-4d15-847d-a594214e21f7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:12.201181 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ae92f-b823-4d15-847d-a594214e21f7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:12.209090 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4f9a9-bc6c-4ee4-a3d8-cdd106c22b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:12.209819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4f9a9-bc6c-4ee4-a3d8-cdd106c22b45 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:12.210092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4f9a9-bc6c-4ee4-a3d8-cdd106c22b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:12.210227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4f9a9-bc6c-4ee4-a3d8-cdd106c22b45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:12.211544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4f9a9-bc6c-4ee4-a3d8-cdd106c22b45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:00:12.212024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4f9a9-bc6c-4ee4-a3d8-cdd106c22b45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:12.213781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4f9a9-bc6c-4ee4-a3d8-cdd106c22b45 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:12.231151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1098/2177] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:00:12.243002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d614df-292f-4968-8ec3-185b6dccb1c7 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:12.248553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d614df-292f-4968-8ec3-185b6dccb1c7 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:12.255119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b289d-8344-482d-8cae-987779611f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:12.256468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b289d-8344-482d-8cae-987779611f2d admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:12.256468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b289d-8344-482d-8cae-987779611f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:12.256468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b289d-8344-482d-8cae-987779611f2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:12.257596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b289d-8344-482d-8cae-987779611f2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:12.260008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b289d-8344-482d-8cae-987779611f2d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:12.314288 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4f9a9-bc6c-4ee4-a3d8-cdd106c22b45 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:12.317325 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e1c4f9a9-bc6c-4ee4-a3d8-cdd106c22b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:12.319373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1080/2178] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:12 2022] POST /identity/v3/projects => generated 377 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:12.323990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b289d-8344-482d-8cae-987779611f2d admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:12.350768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4f9a9-bc6c-4ee4-a3d8-cdd106c22b45 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:12.434509 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:12.445642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b289d-8344-482d-8cae-987779611f2d admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:12.451375 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac4259-7695-4248-9706-18f224b1fe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:12.452077 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac4259-7695-4248-9706-18f224b1fe80 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:12.452823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac4259-7695-4248-9706-18f224b1fe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:12.452823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac4259-7695-4248-9706-18f224b1fe80 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:12.453324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac4259-7695-4248-9706-18f224b1fe80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:12.455327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac4259-7695-4248-9706-18f224b1fe80 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:12.481058 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1081/2179] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:12 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:12.486464 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac4259-7695-4248-9706-18f224b1fe80 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:12.496777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4020cdd-eb6f-43d4-9dd0-e8aa14e6926b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:12.497696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4020cdd-eb6f-43d4-9dd0-e8aa14e6926b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:12.497973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4020cdd-eb6f-43d4-9dd0-e8aa14e6926b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:12.498246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4020cdd-eb6f-43d4-9dd0-e8aa14e6926b admin admin] PATH_INFO: `/v3/users/8f35faab058042198dde279a7429265b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:12.499601 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4020cdd-eb6f-43d4-9dd0-e8aa14e6926b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f35faab058042198dde279a7429265b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:12.501578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4020cdd-eb6f-43d4-9dd0-e8aa14e6926b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:12.509016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac4259-7695-4248-9706-18f224b1fe80 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:12.514529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8207472-ca88-4f37-a99e-4dc516d8cba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:12.515359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207472-ca88-4f37-a99e-4dc516d8cba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:12.515640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207472-ca88-4f37-a99e-4dc516d8cba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:12.515937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207472-ca88-4f37-a99e-4dc516d8cba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:12.516663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8207472-ca88-4f37-a99e-4dc516d8cba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:12.518757 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8207472-ca88-4f37-a99e-4dc516d8cba1 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:12.530975 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1082/2180] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 10:00:12.536342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8207472-ca88-4f37-a99e-4dc516d8cba1 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:12.548405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8207472-ca88-4f37-a99e-4dc516d8cba1 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:12.556548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26774f-6124-48cd-ad46-f75afc36fcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:12.557264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26774f-6124-48cd-ad46-f75afc36fcd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:12.557604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26774f-6124-48cd-ad46-f75afc36fcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:12.557921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26774f-6124-48cd-ad46-f75afc36fcd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:12.559588 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26774f-6124-48cd-ad46-f75afc36fcd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:12.560398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e4020cdd-eb6f-43d4-9dd0-e8aa14e6926b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f35faab058042198dde279a7429265b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:12.561728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26774f-6124-48cd-ad46-f75afc36fcd4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:12.576125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e4020cdd-eb6f-43d4-9dd0-e8aa14e6926b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f35faab058042198dde279a7429265b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:12.577083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e4020cdd-eb6f-43d4-9dd0-e8aa14e6926b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f35faab058042198dde279a7429265b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:12.589786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1083/2181] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:12.593629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e4020cdd-eb6f-43d4-9dd0-e8aa14e6926b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f35faab058042198dde279a7429265b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:12.597538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e4020cdd-eb6f-43d4-9dd0-e8aa14e6926b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f35faab058042198dde279a7429265b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:12.599479 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26774f-6124-48cd-ad46-f75afc36fcd4 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:12.608623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8f26774f-6124-48cd-ad46-f75afc36fcd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:12.623858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1099/2182] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:12 2022] DELETE /identity/v3/users/8f35faab058042198dde279a7429265b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:00:12.625494 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26774f-6124-48cd-ad46-f75afc36fcd4 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:12.642779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:12.737263 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107a4d4-f854-4f4e-8f4a-37bb066cd24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:12.738468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107a4d4-f854-4f4e-8f4a-37bb066cd24c admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:12.738781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107a4d4-f854-4f4e-8f4a-37bb066cd24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:12.739056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107a4d4-f854-4f4e-8f4a-37bb066cd24c admin admin] PATH_INFO: `/v3/projects/8a5be4e0c5b94d8286b65e10106ed14e/users/bff3902f6f4049e7ae49640824a461b7/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:12.759829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107a4d4-f854-4f4e-8f4a-37bb066cd24c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5be4e0c5b94d8286b65e10106ed14e, user_id=bff3902f6f4049e7ae49640824a461b7, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:12.765744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107a4d4-f854-4f4e-8f4a-37bb066cd24c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:12.786529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1084/2183] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:12 2022] PUT /identity/v3/projects/8a5be4e0c5b94d8286b65e10106ed14e/users/bff3902f6f4049e7ae49640824a461b7/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:12.803550 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff85e60-ed28-4214-ae5f-a863b747108a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:12.803890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff85e60-ed28-4214-ae5f-a863b747108a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:12.804106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff85e60-ed28-4214-ae5f-a863b747108a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:12.835255 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4ff85e60-ed28-4214-ae5f-a863b747108a None None] MFA Rules not processed for user `bff3902f6f4049e7ae49640824a461b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:12.837024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4ff85e60-ed28-4214-ae5f-a863b747108a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:12.884375 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107a4d4-f854-4f4e-8f4a-37bb066cd24c admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:12.886214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f107a4d4-f854-4f4e-8f4a-37bb066cd24c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:12.969070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1100/2184] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:12.979864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff85e60-ed28-4214-ae5f-a863b747108a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:13.000257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107a4d4-f854-4f4e-8f4a-37bb066cd24c admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:13.005647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:13.006535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:13.006848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:13.007219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] PATH_INFO: `/v3/projects/8c1f850b21dd4b048311c653945ddd55` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:13.008663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c1f850b21dd4b048311c653945ddd55)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:13.010785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:13.053913 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:13.090774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff85e60-ed28-4214-ae5f-a863b747108a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:13.090972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c1f850b21dd4b048311c653945ddd55', 'request_id': 'req-ac01c11e-0110-429d-9600-1459580fd34b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:13.100419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c1f850b21dd4b048311c653945ddd55', 'request_id': 'req-ac01c11e-0110-429d-9600-1459580fd34b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:13.110684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c1f850b21dd4b048311c653945ddd55', 'request_id': 'req-ac01c11e-0110-429d-9600-1459580fd34b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:13.126828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] The token cache is being invalidate because project 8c1f850b21dd4b048311c653945ddd55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:13.127136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:13.132831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:13.135098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1085/2185] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:12 2022] DELETE /identity/v3/projects/8c1f850b21dd4b048311c653945ddd55 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:13.138961 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:13.141589 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:13.179378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453c699-434d-469f-8257-7dee69b22e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:13.179945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453c699-434d-469f-8257-7dee69b22e10 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:13.180224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453c699-434d-469f-8257-7dee69b22e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:13.180490 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453c699-434d-469f-8257-7dee69b22e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:13.181198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453c699-434d-469f-8257-7dee69b22e10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:13.183189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453c699-434d-469f-8257-7dee69b22e10 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:13.207020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:13.217455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1101/2186] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:12 2022] POST /identity/v3/projects => generated 379 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:00:13.222600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453c699-434d-469f-8257-7dee69b22e10 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:13.228091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1453c699-434d-469f-8257-7dee69b22e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:13.261325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01c11e-0110-429d-9600-1459580fd34b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:13.320473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453c699-434d-469f-8257-7dee69b22e10 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:13.325572 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c8bc1-c26c-46cd-8c1d-76bc426a0de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:13.326269 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8bc1-c26c-46cd-8c1d-76bc426a0de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:13.326553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8bc1-c26c-46cd-8c1d-76bc426a0de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:13.326945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8bc1-c26c-46cd-8c1d-76bc426a0de0 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:13.327906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8bc1-c26c-46cd-8c1d-76bc426a0de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:13.330103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8bc1-c26c-46cd-8c1d-76bc426a0de0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:13.335753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82869d75-c37e-4553-93ff-b32af6b554b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:13.337370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82869d75-c37e-4553-93ff-b32af6b554b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:13.337709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82869d75-c37e-4553-93ff-b32af6b554b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:13.338044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82869d75-c37e-4553-93ff-b32af6b554b3 admin admin] PATH_INFO: `/v3/users/360d89d785164e0f9b8be6795339482a` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:13.339587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82869d75-c37e-4553-93ff-b32af6b554b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=360d89d785164e0f9b8be6795339482a)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:13.342361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82869d75-c37e-4553-93ff-b32af6b554b3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:13.359522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1102/2187] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:13 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:13.366233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c8bc1-c26c-46cd-8c1d-76bc426a0de0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:13.396981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c8bc1-c26c-46cd-8c1d-76bc426a0de0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:13.413758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa59ea5-1c2e-4e6a-90da-888c9a8bf142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:13.413758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa59ea5-1c2e-4e6a-90da-888c9a8bf142 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:13.413758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa59ea5-1c2e-4e6a-90da-888c9a8bf142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:13.413758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa59ea5-1c2e-4e6a-90da-888c9a8bf142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:13.414409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa59ea5-1c2e-4e6a-90da-888c9a8bf142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:13.419247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa59ea5-1c2e-4e6a-90da-888c9a8bf142 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:13.435956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1103/2188] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:13.443822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa59ea5-1c2e-4e6a-90da-888c9a8bf142 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:13.463489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-82869d75-c37e-4553-93ff-b32af6b554b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '360d89d785164e0f9b8be6795339482a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:13.464881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa59ea5-1c2e-4e6a-90da-888c9a8bf142 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:13.473745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7f969-1c88-424e-b1cb-5e661b57611d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:13.474557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7f969-1c88-424e-b1cb-5e661b57611d admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:13.475070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7f969-1c88-424e-b1cb-5e661b57611d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:13.475350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7f969-1c88-424e-b1cb-5e661b57611d admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:13.476190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7f969-1c88-424e-b1cb-5e661b57611d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:13.481486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7f969-1c88-424e-b1cb-5e661b57611d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:13.489782 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-82869d75-c37e-4553-93ff-b32af6b554b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '360d89d785164e0f9b8be6795339482a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:13.490937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-82869d75-c37e-4553-93ff-b32af6b554b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '360d89d785164e0f9b8be6795339482a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:13.491757 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-82869d75-c37e-4553-93ff-b32af6b554b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '360d89d785164e0f9b8be6795339482a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:13.502050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1104/2189] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:13.504603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-82869d75-c37e-4553-93ff-b32af6b554b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '360d89d785164e0f9b8be6795339482a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:13.506925 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7f969-1c88-424e-b1cb-5e661b57611d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:13.508841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-63d7f969-1c88-424e-b1cb-5e661b57611d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:13.544847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7f969-1c88-424e-b1cb-5e661b57611d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:13.550314 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1086/2190] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:13 2022] DELETE /identity/v3/users/360d89d785164e0f9b8be6795339482a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:00:13.605833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:13.632329 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d2d4e-1c9f-4feb-b815-04e446282025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:13.633274 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d2d4e-1c9f-4feb-b815-04e446282025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:13.633556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d2d4e-1c9f-4feb-b815-04e446282025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:13.633919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d2d4e-1c9f-4feb-b815-04e446282025 admin admin] PATH_INFO: `/v3/projects/6d3a9b88bdd54a7e81a272ff13661e53/users/7c75538031ca4a4e810b9d50fa5646cc/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:13.653412 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d2d4e-1c9f-4feb-b815-04e446282025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3a9b88bdd54a7e81a272ff13661e53, user_id=7c75538031ca4a4e810b9d50fa5646cc, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:13.656255 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d2d4e-1c9f-4feb-b815-04e446282025 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:13.673525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1105/2191] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:13 2022] PUT /identity/v3/projects/6d3a9b88bdd54a7e81a272ff13661e53/users/7c75538031ca4a4e810b9d50fa5646cc/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:00:13.686813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71439f-1b34-44d6-9e60-fbb38d9740eb None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:13.686813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71439f-1b34-44d6-9e60-fbb38d9740eb None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:13.686813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71439f-1b34-44d6-9e60-fbb38d9740eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:13.722549 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-7a71439f-1b34-44d6-9e60-fbb38d9740eb None None] MFA Rules not processed for user `7c75538031ca4a4e810b9d50fa5646cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:13.724203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7a71439f-1b34-44d6-9e60-fbb38d9740eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:13.779607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d2d4e-1c9f-4feb-b815-04e446282025 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:13.782491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-812d2d4e-1c9f-4feb-b815-04e446282025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:13.841398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:13.887962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1087/2192] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:13.892627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71439f-1b34-44d6-9e60-fbb38d9740eb None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:13.939043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d2d4e-1c9f-4feb-b815-04e446282025 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:13.946668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:13.948066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:13.948471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:13.948792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] PATH_INFO: `/v3/projects/9d0e7518ffb1418ea24b1507d229f3c4` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:13.950131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0e7518ffb1418ea24b1507d229f3c4)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:13.956772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:13.987890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71439f-1b34-44d6-9e60-fbb38d9740eb None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:13.993320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeebd5a-5334-4969-ad5c-9403587ce9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:13.994116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeebd5a-5334-4969-ad5c-9403587ce9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:13.994499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeebd5a-5334-4969-ad5c-9403587ce9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:13.994905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeebd5a-5334-4969-ad5c-9403587ce9c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:13.995711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeebd5a-5334-4969-ad5c-9403587ce9c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:13.997903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeebd5a-5334-4969-ad5c-9403587ce9c6 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.028203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1088/2193] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:13 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:14.032472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeebd5a-5334-4969-ad5c-9403587ce9c6 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.032905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0e7518ffb1418ea24b1507d229f3c4', 'request_id': 'req-d529036d-3498-4b21-bbec-73049494b575'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:14.044982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0e7518ffb1418ea24b1507d229f3c4', 'request_id': 'req-d529036d-3498-4b21-bbec-73049494b575'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:14.052710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0e7518ffb1418ea24b1507d229f3c4', 'request_id': 'req-d529036d-3498-4b21-bbec-73049494b575'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:14.067569 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] The token cache is being invalidate because project 9d0e7518ffb1418ea24b1507d229f3c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:14.067857 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:14.068544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d529036d-3498-4b21-bbec-73049494b575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:14.070292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1106/2194] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:13 2022] DELETE /identity/v3/projects/9d0e7518ffb1418ea24b1507d229f3c4 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:14.077211 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00d03f-b0b6-4f96-8a95-14f8176302e9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.077757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00d03f-b0b6-4f96-8a95-14f8176302e9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.078225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00d03f-b0b6-4f96-8a95-14f8176302e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.100937 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1c00d03f-b0b6-4f96-8a95-14f8176302e9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:14.102489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1c00d03f-b0b6-4f96-8a95-14f8176302e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:14.104619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeebd5a-5334-4969-ad5c-9403587ce9c6 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.109456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325bfee-1297-4a21-b0a9-c2c95fd69bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.110276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325bfee-1297-4a21-b0a9-c2c95fd69bff admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.110575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325bfee-1297-4a21-b0a9-c2c95fd69bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.110861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325bfee-1297-4a21-b0a9-c2c95fd69bff admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.111382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1107/2195] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:14.111565 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325bfee-1297-4a21-b0a9-c2c95fd69bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.113977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325bfee-1297-4a21-b0a9-c2c95fd69bff admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.119040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96553073-9882-4ddf-b38c-ff494f9581de None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.119426 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96553073-9882-4ddf-b38c-ff494f9581de None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.119799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96553073-9882-4ddf-b38c-ff494f9581de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.139543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1089/2196] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:14 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:00:14.141322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-96553073-9882-4ddf-b38c-ff494f9581de None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:14.142637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-96553073-9882-4ddf-b38c-ff494f9581de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:14.143886 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325bfee-1297-4a21-b0a9-c2c95fd69bff admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.145576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4325bfee-1297-4a21-b0a9-c2c95fd69bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:14.151986 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1108/2197] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:00:14.155958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96553073-9882-4ddf-b38c-ff494f9581de None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.158902 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325bfee-1297-4a21-b0a9-c2c95fd69bff admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.165816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e2efa-8d24-4990-a375-fe415bec24cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.166512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e2efa-8d24-4990-a375-fe415bec24cb admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.166755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96553073-9882-4ddf-b38c-ff494f9581de None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.166862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e2efa-8d24-4990-a375-fe415bec24cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.167109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e2efa-8d24-4990-a375-fe415bec24cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.167852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e2efa-8d24-4990-a375-fe415bec24cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.169477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e2efa-8d24-4990-a375-fe415bec24cb admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.172053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd336f-ffd9-4e9f-91f3-fd0bcf8845de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.172809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd336f-ffd9-4e9f-91f3-fd0bcf8845de admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.173154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd336f-ffd9-4e9f-91f3-fd0bcf8845de admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.173486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd336f-ffd9-4e9f-91f3-fd0bcf8845de admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.174428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd336f-ffd9-4e9f-91f3-fd0bcf8845de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:00:14.175078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd336f-ffd9-4e9f-91f3-fd0bcf8845de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.177036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd336f-ffd9-4e9f-91f3-fd0bcf8845de admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.184150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1090/2198] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 10:00:14.185799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e2efa-8d24-4990-a375-fe415bec24cb admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.194672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1109/2199] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 10:00:14.197953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd336f-ffd9-4e9f-91f3-fd0bcf8845de admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.207432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e2efa-8d24-4990-a375-fe415bec24cb admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.209519 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd336f-ffd9-4e9f-91f3-fd0bcf8845de admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.213107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94983d-9a25-42c6-b299-8d41f32c27d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.214019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94983d-9a25-42c6-b299-8d41f32c27d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.214299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94983d-9a25-42c6-b299-8d41f32c27d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.214585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94983d-9a25-42c6-b299-8d41f32c27d7 admin admin] PATH_INFO: `/v3/projects/9ee87f1249c945439e2ce3ca2a9f57d9/users/3c7e16d393ac4c959eb55cfa7bf07bc6/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.215689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c1684-db9e-4edd-ac19-b07d16ce2833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.216354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c1684-db9e-4edd-ac19-b07d16ce2833 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.216632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c1684-db9e-4edd-ac19-b07d16ce2833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.217054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c1684-db9e-4edd-ac19-b07d16ce2833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.217681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c1684-db9e-4edd-ac19-b07d16ce2833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.219670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c1684-db9e-4edd-ac19-b07d16ce2833 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.234810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94983d-9a25-42c6-b299-8d41f32c27d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee87f1249c945439e2ce3ca2a9f57d9, user_id=3c7e16d393ac4c959eb55cfa7bf07bc6, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.238309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94983d-9a25-42c6-b299-8d41f32c27d7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.254635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1110/2200] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:14 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:14.255778 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1091/2201] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:14 2022] PUT /identity/v3/projects/9ee87f1249c945439e2ce3ca2a9f57d9/users/3c7e16d393ac4c959eb55cfa7bf07bc6/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:14.258613 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c1684-db9e-4edd-ac19-b07d16ce2833 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.259920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94983d-9a25-42c6-b299-8d41f32c27d7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.264534 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:14.298761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:14.333114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c1684-db9e-4edd-ac19-b07d16ce2833 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.333452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94983d-9a25-42c6-b299-8d41f32c27d7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.338241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcb5e5-f8b1-45e8-afda-ddc3086f756b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.338510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876339a-e65c-42a9-a629-8083d9f72a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.338938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb5e5-f8b1-45e8-afda-ddc3086f756b admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.339223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb5e5-f8b1-45e8-afda-ddc3086f756b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.339349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876339a-e65c-42a9-a629-8083d9f72a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.339485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb5e5-f8b1-45e8-afda-ddc3086f756b admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.339577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876339a-e65c-42a9-a629-8083d9f72a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.339764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876339a-e65c-42a9-a629-8083d9f72a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.340235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcb5e5-f8b1-45e8-afda-ddc3086f756b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.340493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876339a-e65c-42a9-a629-8083d9f72a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.342129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876339a-e65c-42a9-a629-8083d9f72a53 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.342273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcb5e5-f8b1-45e8-afda-ddc3086f756b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.353360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1092/2202] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:14.357870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876339a-e65c-42a9-a629-8083d9f72a53 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.365433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1111/2203] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:14 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:14.366481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcb5e5-f8b1-45e8-afda-ddc3086f756b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.368386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876339a-e65c-42a9-a629-8083d9f72a53 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.374453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e65b34-1039-4e85-bca4-839d11e03e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.375393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65b34-1039-4e85-bca4-839d11e03e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.375677 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65b34-1039-4e85-bca4-839d11e03e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.375982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65b34-1039-4e85-bca4-839d11e03e48 admin admin] PATH_INFO: `/v3/projects/9ee87f1249c945439e2ce3ca2a9f57d9/users/3c7e16d393ac4c959eb55cfa7bf07bc6/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.376188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcb5e5-f8b1-45e8-afda-ddc3086f756b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.378099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e65b34-1039-4e85-bca4-839d11e03e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee87f1249c945439e2ce3ca2a9f57d9, user_id=3c7e16d393ac4c959eb55cfa7bf07bc6, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.381162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e65b34-1039-4e85-bca4-839d11e03e48 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.381275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f625b-d161-4d76-84ff-09ba911c7a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.381838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f625b-d161-4d76-84ff-09ba911c7a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.382112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f625b-d161-4d76-84ff-09ba911c7a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.382388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f625b-d161-4d76-84ff-09ba911c7a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.383137 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f625b-d161-4d76-84ff-09ba911c7a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.384785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f625b-d161-4d76-84ff-09ba911c7a3a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.393049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1093/2204] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:14 2022] PUT /identity/v3/projects/9ee87f1249c945439e2ce3ca2a9f57d9/users/3c7e16d393ac4c959eb55cfa7bf07bc6/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:00:14.396457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1112/2205] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:14.401706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e65b34-1039-4e85-bca4-839d11e03e48 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.404042 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9fa35-913e-45e6-9831-25f9831e0f74 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.404361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9fa35-913e-45e6-9831-25f9831e0f74 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.404766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9fa35-913e-45e6-9831-25f9831e0f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.435296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3fd9fa35-913e-45e6-9831-25f9831e0f74 None None] MFA Rules not processed for user `3c7e16d393ac4c959eb55cfa7bf07bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:14.436776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3fd9fa35-913e-45e6-9831-25f9831e0f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:14.468808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e65b34-1039-4e85-bca4-839d11e03e48 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.479194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045aa559-a190-4ffb-80e4-e3b5ee3fe9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.479194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045aa559-a190-4ffb-80e4-e3b5ee3fe9ab admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.479194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045aa559-a190-4ffb-80e4-e3b5ee3fe9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.479194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045aa559-a190-4ffb-80e4-e3b5ee3fe9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.479775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045aa559-a190-4ffb-80e4-e3b5ee3fe9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.479775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045aa559-a190-4ffb-80e4-e3b5ee3fe9ab admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.489181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1094/2206] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:14.493266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045aa559-a190-4ffb-80e4-e3b5ee3fe9ab admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.503753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045aa559-a190-4ffb-80e4-e3b5ee3fe9ab admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.508778 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0a307-b10b-4657-8d29-93426c41d84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.509660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a307-b10b-4657-8d29-93426c41d84b admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.509937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a307-b10b-4657-8d29-93426c41d84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.510207 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a307-b10b-4657-8d29-93426c41d84b admin admin] PATH_INFO: `/v3/projects/2f6531062b964295a92c3bb00cceb504/users/fd47c784ffb04bada9a16739ba42e2ef/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.511959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1113/2207] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:00:14.523444 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9fa35-913e-45e6-9831-25f9831e0f74 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.524953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3fd9fa35-913e-45e6-9831-25f9831e0f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:14.526813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0a307-b10b-4657-8d29-93426c41d84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f6531062b964295a92c3bb00cceb504, user_id=fd47c784ffb04bada9a16739ba42e2ef, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.529533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0a307-b10b-4657-8d29-93426c41d84b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.549382 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1095/2208] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:14 2022] PUT /identity/v3/projects/2f6531062b964295a92c3bb00cceb504/users/fd47c784ffb04bada9a16739ba42e2ef/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:14.568618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fc6ff-504e-4211-ba2b-aa3ccb7e02eb None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.569096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fc6ff-504e-4211-ba2b-aa3ccb7e02eb None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.569520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fc6ff-504e-4211-ba2b-aa3ccb7e02eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.610216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-f71fc6ff-504e-4211-ba2b-aa3ccb7e02eb None None] MFA Rules not processed for user `fd47c784ffb04bada9a16739ba42e2ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:14.611531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f71fc6ff-504e-4211-ba2b-aa3ccb7e02eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:14.613974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9fa35-913e-45e6-9831-25f9831e0f74 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.637349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:14.676548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788d5bf-67b0-46de-8b82-a73e9cd47c8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.677231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788d5bf-67b0-46de-8b82-a73e9cd47c8f service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.677492 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788d5bf-67b0-46de-8b82-a73e9cd47c8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.677766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788d5bf-67b0-46de-8b82-a73e9cd47c8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.695712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1096/2209] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:14.709847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fc6ff-504e-4211-ba2b-aa3ccb7e02eb None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.712588 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f71fc6ff-504e-4211-ba2b-aa3ccb7e02eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:14.738610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fc6ff-504e-4211-ba2b-aa3ccb7e02eb None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.743119 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3c303-767a-45a8-b460-d6ba83abfc81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.743815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3c303-767a-45a8-b460-d6ba83abfc81 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.744079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3c303-767a-45a8-b460-d6ba83abfc81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.744509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3c303-767a-45a8-b460-d6ba83abfc81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.753533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788d5bf-67b0-46de-8b82-a73e9cd47c8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.754772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3c303-767a-45a8-b460-d6ba83abfc81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.755293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788d5bf-67b0-46de-8b82-a73e9cd47c8f service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.756382 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3c303-767a-45a8-b460-d6ba83abfc81 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.760572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:14.767879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1114/2210] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:00:14.777729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1097/2211] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:00:14.797195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788d5bf-67b0-46de-8b82-a73e9cd47c8f service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.817217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3c303-767a-45a8-b460-d6ba83abfc81 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:14.830880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3c303-767a-45a8-b460-d6ba83abfc81 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.835423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95034923-45f3-4851-a4e6-759275bd28af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.836573 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95034923-45f3-4851-a4e6-759275bd28af service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.836849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95034923-45f3-4851-a4e6-759275bd28af service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.837126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95034923-45f3-4851-a4e6-759275bd28af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.839306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-95034923-45f3-4851-a4e6-759275bd28af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:14.876098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788d5bf-67b0-46de-8b82-a73e9cd47c8f service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:14.881597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:14.882547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.882880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.883151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] PATH_INFO: `/v3/users/fd47c784ffb04bada9a16739ba42e2ef` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.884375 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd47c784ffb04bada9a16739ba42e2ef)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.886393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.902283 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95034923-45f3-4851-a4e6-759275bd28af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:14.903774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95034923-45f3-4851-a4e6-759275bd28af service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:14.913265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1098/2212] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:00:14.937883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd47c784ffb04bada9a16739ba42e2ef'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:14.950361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd47c784ffb04bada9a16739ba42e2ef'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:14.951123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd47c784ffb04bada9a16739ba42e2ef'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:14.958515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd47c784ffb04bada9a16739ba42e2ef'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:14.959303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd47c784ffb04bada9a16739ba42e2ef'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:14.963103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:14.976167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1115/2213] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:14 2022] DELETE /identity/v3/users/fd47c784ffb04bada9a16739ba42e2ef => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:14.980791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9c115-def2-4b67-a1fb-17af5a179147 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:14.981080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9c115-def2-4b67-a1fb-17af5a179147 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:14.981378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9c115-def2-4b67-a1fb-17af5a179147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:14.999742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2ee9c115-def2-4b67-a1fb-17af5a179147 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:15.000933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2ee9c115-def2-4b67-a1fb-17af5a179147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:15.064150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1099/2214] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:15.075737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:15.080113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:15.160847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd65257-335c-45dd-b10b-a45e8fb68605 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:15.165264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47cce6-6db0-4649-8e9f-57a36279791a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:15.165988 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47cce6-6db0-4649-8e9f-57a36279791a service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:15.166269 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47cce6-6db0-4649-8e9f-57a36279791a service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:15.166543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47cce6-6db0-4649-8e9f-57a36279791a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:15.176735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47cce6-6db0-4649-8e9f-57a36279791a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:15.178217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47cce6-6db0-4649-8e9f-57a36279791a service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:15.188452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1116/2215] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:15.506468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9c115-def2-4b67-a1fb-17af5a179147 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:15.508020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2ee9c115-def2-4b67-a1fb-17af5a179147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:15.522810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9c115-def2-4b67-a1fb-17af5a179147 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:15.528195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:15.529124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:15.529401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:15.529672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] PATH_INFO: `/v3/projects/2f6531062b964295a92c3bb00cceb504` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:15.530911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f6531062b964295a92c3bb00cceb504)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:15.533731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:15.585041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f6531062b964295a92c3bb00cceb504', 'request_id': 'req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:15.594039 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f6531062b964295a92c3bb00cceb504', 'request_id': 'req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:15.601957 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f6531062b964295a92c3bb00cceb504', 'request_id': 'req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:15.616032 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] The token cache is being invalidate because project 2f6531062b964295a92c3bb00cceb504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:15.616405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:15.617221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1fee55dc-cb59-4dbe-9ecc-5a9b38b1004b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:15.622505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1100/2216] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:15 2022] DELETE /identity/v3/projects/2f6531062b964295a92c3bb00cceb504 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:15.624156 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04232-d337-4f52-85be-d2d6751c03a0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:15.624843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04232-d337-4f52-85be-d2d6751c03a0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:15.625131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04232-d337-4f52-85be-d2d6751c03a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:15.654978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4ac04232-d337-4f52-85be-d2d6751c03a0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:15.656222 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4ac04232-d337-4f52-85be-d2d6751c03a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:15.715064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1117/2217] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:00:15.726415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0029e9-21f2-4b13-9562-bcb08c23553b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:15.726903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0029e9-21f2-4b13-9562-bcb08c23553b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:15.727289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0029e9-21f2-4b13-9562-bcb08c23553b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:15.749584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6a0029e9-21f2-4b13-9562-bcb08c23553b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:15.750822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6a0029e9-21f2-4b13-9562-bcb08c23553b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:15.761729 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1101/2218] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:15.766333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac04232-d337-4f52-85be-d2d6751c03a0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:15.777862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac04232-d337-4f52-85be-d2d6751c03a0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:15.783460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cb5fc-6a24-496e-9ed2-6540abaa2c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:15.784785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cb5fc-6a24-496e-9ed2-6540abaa2c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:15.785059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cb5fc-6a24-496e-9ed2-6540abaa2c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:15.785331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cb5fc-6a24-496e-9ed2-6540abaa2c03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:15.786069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cb5fc-6a24-496e-9ed2-6540abaa2c03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:00:15.786358 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cb5fc-6a24-496e-9ed2-6540abaa2c03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:15.788030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cb5fc-6a24-496e-9ed2-6540abaa2c03 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:15.802211 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1118/2219] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:00:15.806605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0029e9-21f2-4b13-9562-bcb08c23553b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:15.815845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0029e9-21f2-4b13-9562-bcb08c23553b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:15.821439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b3268-7708-4dbf-8848-66d5f4a36b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:15.822100 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b3268-7708-4dbf-8848-66d5f4a36b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:15.822387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b3268-7708-4dbf-8848-66d5f4a36b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:15.822777 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b3268-7708-4dbf-8848-66d5f4a36b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:15.823440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b3268-7708-4dbf-8848-66d5f4a36b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:15.825350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b3268-7708-4dbf-8848-66d5f4a36b81 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:15.840141 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:15.864058 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1102/2220] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:15 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:15.865330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cb5fc-6a24-496e-9ed2-6540abaa2c03 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:15.879179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cb5fc-6a24-496e-9ed2-6540abaa2c03 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:15.949197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badca43e-00ec-429c-a075-35e9283b12ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:15.949969 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badca43e-00ec-429c-a075-35e9283b12ac admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:15.950321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badca43e-00ec-429c-a075-35e9283b12ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:15.950731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badca43e-00ec-429c-a075-35e9283b12ac admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:15.951507 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badca43e-00ec-429c-a075-35e9283b12ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:15.953526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badca43e-00ec-429c-a075-35e9283b12ac admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:15.978792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1119/2221] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:15 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:15.982906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b3268-7708-4dbf-8848-66d5f4a36b81 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:15.993747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b3268-7708-4dbf-8848-66d5f4a36b81 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:15.998479 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e1f09-98be-4be0-9bef-8a70707189d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:15.999178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e1f09-98be-4be0-9bef-8a70707189d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:15.999462 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e1f09-98be-4be0-9bef-8a70707189d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:15.999738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e1f09-98be-4be0-9bef-8a70707189d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:16.000469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e1f09-98be-4be0-9bef-8a70707189d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:16.002074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e1f09-98be-4be0-9bef-8a70707189d8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:16.013578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1103/2222] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:16.017777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badca43e-00ec-429c-a075-35e9283b12ac admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:16.028753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badca43e-00ec-429c-a075-35e9283b12ac admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:16.034540 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc8a22-f43e-4fe1-a0b7-19ca39fc4794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:16.035191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc8a22-f43e-4fe1-a0b7-19ca39fc4794 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:16.035472 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc8a22-f43e-4fe1-a0b7-19ca39fc4794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:16.035745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc8a22-f43e-4fe1-a0b7-19ca39fc4794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:16.036474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc8a22-f43e-4fe1-a0b7-19ca39fc4794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:16.038083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc8a22-f43e-4fe1-a0b7-19ca39fc4794 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:16.049037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1120/2223] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:16.053727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e1f09-98be-4be0-9bef-8a70707189d8 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:16.066296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e1f09-98be-4be0-9bef-8a70707189d8 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:16.071289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b5445-6b03-435c-b73b-4d6f18dbb439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:16.072160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b5445-6b03-435c-b73b-4d6f18dbb439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:16.072429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b5445-6b03-435c-b73b-4d6f18dbb439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:16.072699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b5445-6b03-435c-b73b-4d6f18dbb439 admin admin] PATH_INFO: `/v3/projects/a35c72dcd7fa494c97660b578df6a462/users/f78c7829473647929af9a48031814201/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:16.090062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b5445-6b03-435c-b73b-4d6f18dbb439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a35c72dcd7fa494c97660b578df6a462, user_id=f78c7829473647929af9a48031814201, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:16.092817 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b5445-6b03-435c-b73b-4d6f18dbb439 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:16.107343 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1104/2224] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:16 2022] PUT /identity/v3/projects/a35c72dcd7fa494c97660b578df6a462/users/f78c7829473647929af9a48031814201/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:16.118743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0259a-b237-46a5-965b-58258dd4e342 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:16.119037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0259a-b237-46a5-965b-58258dd4e342 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:16.119318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0259a-b237-46a5-965b-58258dd4e342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:16.125642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:16.152401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dac0259a-b237-46a5-965b-58258dd4e342 None None] MFA Rules not processed for user `f78c7829473647929af9a48031814201`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:16.154255 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dac0259a-b237-46a5-965b-58258dd4e342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:16.234015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1121/2225] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:16.248509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b5445-6b03-435c-b73b-4d6f18dbb439 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:16.250946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2a5b5445-6b03-435c-b73b-4d6f18dbb439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:16.317457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b5445-6b03-435c-b73b-4d6f18dbb439 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:16.321721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e03c3a-0310-471b-a95c-7348d6374764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:16.322364 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e03c3a-0310-471b-a95c-7348d6374764 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:16.322835 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e03c3a-0310-471b-a95c-7348d6374764 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:16.323107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e03c3a-0310-471b-a95c-7348d6374764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:16.332464 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e03c3a-0310-471b-a95c-7348d6374764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:16.333948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e03c3a-0310-471b-a95c-7348d6374764 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:16.343796 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1105/2226] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:18.166888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b170f-49e7-4b92-b2f0-bf3958987ea5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:18.167504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b170f-49e7-4b92-b2f0-bf3958987ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:18.167504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b170f-49e7-4b92-b2f0-bf3958987ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:18.186349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c50b170f-49e7-4b92-b2f0-bf3958987ea5 None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:18.187642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c50b170f-49e7-4b92-b2f0-bf3958987ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:18.249596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1122/2227] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 10:00:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:18.299574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e03c3a-0310-471b-a95c-7348d6374764 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:18.301846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a6e03c3a-0310-471b-a95c-7348d6374764 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:18.378512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e03c3a-0310-471b-a95c-7348d6374764 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:18.384642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a200ad9-3168-42a8-8ebe-615251e5c4af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:18.385306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a200ad9-3168-42a8-8ebe-615251e5c4af service swift] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:18.385581 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a200ad9-3168-42a8-8ebe-615251e5c4af service swift] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:18.385852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a200ad9-3168-42a8-8ebe-615251e5c4af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:18.395118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a200ad9-3168-42a8-8ebe-615251e5c4af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:18.396709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a200ad9-3168-42a8-8ebe-615251e5c4af service swift] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:18.405694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1106/2228] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:00:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:00:22.159944 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b170f-49e7-4b92-b2f0-bf3958987ea5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:22.162041 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c50b170f-49e7-4b92-b2f0-bf3958987ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:22.263107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b170f-49e7-4b92-b2f0-bf3958987ea5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:22.268798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:22.269848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:22.270197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:22.270538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] PATH_INFO: `/v3/users/bff3902f6f4049e7ae49640824a461b7` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:22.272850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff3902f6f4049e7ae49640824a461b7)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:22.275294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:22.328005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:22.339809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff3902f6f4049e7ae49640824a461b7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:22.370670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff3902f6f4049e7ae49640824a461b7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:22.370670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff3902f6f4049e7ae49640824a461b7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:22.392737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff3902f6f4049e7ae49640824a461b7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:22.394808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff3902f6f4049e7ae49640824a461b7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:22.414165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1123/2229] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:22 2022] DELETE /identity/v3/users/bff3902f6f4049e7ae49640824a461b7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:22.421126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432df64b-a199-495d-83af-6373fc84671b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:22.422478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432df64b-a199-495d-83af-6373fc84671b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:22.423164 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432df64b-a199-495d-83af-6373fc84671b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:22.451475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-432df64b-a199-495d-83af-6373fc84671b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:22.453393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-432df64b-a199-495d-83af-6373fc84671b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:22.528541 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:22.554304 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1107/2230] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:22.561773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:22.564114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:22.727197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b265b4d-08e4-48c8-b446-0fa71faa84fa admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:22.733076 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399a1f0-b809-45de-a865-6424577767de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:22.740961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399a1f0-b809-45de-a865-6424577767de service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:22.741672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399a1f0-b809-45de-a865-6424577767de service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:22.741998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399a1f0-b809-45de-a865-6424577767de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:22.760066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399a1f0-b809-45de-a865-6424577767de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:22.761370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399a1f0-b809-45de-a865-6424577767de service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:22.777724 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1124/2231] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:23.268426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432df64b-a199-495d-83af-6373fc84671b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:23.270103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-432df64b-a199-495d-83af-6373fc84671b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:23.285833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432df64b-a199-495d-83af-6373fc84671b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:23.307761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:23.308855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:23.309307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:23.309667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] PATH_INFO: `/v3/projects/8a5be4e0c5b94d8286b65e10106ed14e` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:23.311474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5be4e0c5b94d8286b65e10106ed14e)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:23.313641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:23.413218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5be4e0c5b94d8286b65e10106ed14e', 'request_id': 'req-f152bca9-87b5-4ac3-9041-53619925d4d6'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:23.424297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5be4e0c5b94d8286b65e10106ed14e', 'request_id': 'req-f152bca9-87b5-4ac3-9041-53619925d4d6'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:23.433403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5be4e0c5b94d8286b65e10106ed14e', 'request_id': 'req-f152bca9-87b5-4ac3-9041-53619925d4d6'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:23.452210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] The token cache is being invalidate because project 8a5be4e0c5b94d8286b65e10106ed14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:23.452633 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:23.453489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:23.455753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1108/2232] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:23 2022] DELETE /identity/v3/projects/8a5be4e0c5b94d8286b65e10106ed14e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:23.461533 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399a1f0-b809-45de-a865-6424577767de service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:23.475866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399a1f0-b809-45de-a865-6424577767de service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:23.560921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:23.562031 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:23.562401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:23.562782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] PATH_INFO: `/v3/users/7c75538031ca4a4e810b9d50fa5646cc` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:23.564468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c75538031ca4a4e810b9d50fa5646cc)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:23.567361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:23.632532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c75538031ca4a4e810b9d50fa5646cc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:23.646160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:23.650543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c75538031ca4a4e810b9d50fa5646cc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:23.651611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c75538031ca4a4e810b9d50fa5646cc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:23.665776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c75538031ca4a4e810b9d50fa5646cc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:23.665776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c75538031ca4a4e810b9d50fa5646cc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:23.682539 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1125/2233] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:23 2022] DELETE /identity/v3/users/7c75538031ca4a4e810b9d50fa5646cc => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:24.177390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:24.187722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:24.238053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152bca9-87b5-4ac3-9041-53619925d4d6 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:24.343835 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:24.344949 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:24.345344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:24.345718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] PATH_INFO: `/v3/projects/6d3a9b88bdd54a7e81a272ff13661e53` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:24.347166 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3a9b88bdd54a7e81a272ff13661e53)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:24.349723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:24.451347 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3a9b88bdd54a7e81a272ff13661e53', 'request_id': 'req-63b4ef07-932d-4018-a5ce-342134f35e65'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:24.468943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3a9b88bdd54a7e81a272ff13661e53', 'request_id': 'req-63b4ef07-932d-4018-a5ce-342134f35e65'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:24.480508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3a9b88bdd54a7e81a272ff13661e53', 'request_id': 'req-63b4ef07-932d-4018-a5ce-342134f35e65'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:24.504709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] The token cache is being invalidate because project 6d3a9b88bdd54a7e81a272ff13661e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:24.505143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:24.508132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:24.510851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1109/2234] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:24 2022] DELETE /identity/v3/projects/6d3a9b88bdd54a7e81a272ff13661e53 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:24.516051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:24.518419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:24.534096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000363af-47fc-40b4-81be-d3eda3d6e811 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:24.623514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174125d-6cae-402d-b1e4-c6139f31b58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:24.624518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174125d-6cae-402d-b1e4-c6139f31b58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:24.624934 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174125d-6cae-402d-b1e4-c6139f31b58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:24.625217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174125d-6cae-402d-b1e4-c6139f31b58f admin admin] PATH_INFO: `/v3/users/3c7e16d393ac4c959eb55cfa7bf07bc6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:24.626406 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174125d-6cae-402d-b1e4-c6139f31b58f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c7e16d393ac4c959eb55cfa7bf07bc6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:24.628531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174125d-6cae-402d-b1e4-c6139f31b58f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:24.691348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3174125d-6cae-402d-b1e4-c6139f31b58f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c7e16d393ac4c959eb55cfa7bf07bc6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:24.710988 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3174125d-6cae-402d-b1e4-c6139f31b58f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c7e16d393ac4c959eb55cfa7bf07bc6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:24.712921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3174125d-6cae-402d-b1e4-c6139f31b58f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c7e16d393ac4c959eb55cfa7bf07bc6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:24.725904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3174125d-6cae-402d-b1e4-c6139f31b58f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c7e16d393ac4c959eb55cfa7bf07bc6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:24.727371 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3174125d-6cae-402d-b1e4-c6139f31b58f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c7e16d393ac4c959eb55cfa7bf07bc6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:24.749522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1126/2235] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:24 2022] DELETE /identity/v3/users/3c7e16d393ac4c959eb55cfa7bf07bc6 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:00:25.156661 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:25.158265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:25.168282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:25.180457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4ef07-932d-4018-a5ce-342134f35e65 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:25.297596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:25.298667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:25.298948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:25.299228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] PATH_INFO: `/v3/projects/9ee87f1249c945439e2ce3ca2a9f57d9` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:25.300728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee87f1249c945439e2ce3ca2a9f57d9)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:25.302733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:25.404513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee87f1249c945439e2ce3ca2a9f57d9', 'request_id': 'req-44803064-7ac7-4a49-a181-e6474aec2582'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:25.416896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee87f1249c945439e2ce3ca2a9f57d9', 'request_id': 'req-44803064-7ac7-4a49-a181-e6474aec2582'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:25.429877 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee87f1249c945439e2ce3ca2a9f57d9', 'request_id': 'req-44803064-7ac7-4a49-a181-e6474aec2582'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:25.449783 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] The token cache is being invalidate because project 9ee87f1249c945439e2ce3ca2a9f57d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:25.452850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:25.454412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-44803064-7ac7-4a49-a181-e6474aec2582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:25.459043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1110/2236] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:25 2022] DELETE /identity/v3/projects/9ee87f1249c945439e2ce3ca2a9f57d9 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:25.467151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e05c9-9dc2-4eca-86d2-ad40eee8384e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:25.467451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e05c9-9dc2-4eca-86d2-ad40eee8384e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:25.467722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e05c9-9dc2-4eca-86d2-ad40eee8384e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:25.495159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5c3e05c9-9dc2-4eca-86d2-ad40eee8384e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:25.507224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5c3e05c9-9dc2-4eca-86d2-ad40eee8384e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:25.592946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:25.622779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1127/2237] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:25.623828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ae85d-c77b-4a5f-ab90-c85ea4b220f7 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:25.624280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ae85d-c77b-4a5f-ab90-c85ea4b220f7 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:25.624713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ae85d-c77b-4a5f-ab90-c85ea4b220f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:25.648471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-200ae85d-c77b-4a5f-ab90-c85ea4b220f7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:25.650508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-200ae85d-c77b-4a5f-ab90-c85ea4b220f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:25.660342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1111/2238] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:00:25.665081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e05c9-9dc2-4eca-86d2-ad40eee8384e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:25.677977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e05c9-9dc2-4eca-86d2-ad40eee8384e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:25.690726 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc7258-3fcf-4031-9e30-b49f4caa0e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:25.691564 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc7258-3fcf-4031-9e30-b49f4caa0e55 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:25.691884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc7258-3fcf-4031-9e30-b49f4caa0e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:25.692149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc7258-3fcf-4031-9e30-b49f4caa0e55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:25.692926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc7258-3fcf-4031-9e30-b49f4caa0e55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:00:25.693404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc7258-3fcf-4031-9e30-b49f4caa0e55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:25.696040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc7258-3fcf-4031-9e30-b49f4caa0e55 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:25.723117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1128/2239] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:00:25.723889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ae85d-c77b-4a5f-ab90-c85ea4b220f7 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:25.734438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ae85d-c77b-4a5f-ab90-c85ea4b220f7 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:25.740023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac2c27-f910-49a4-921c-439dff6006cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:25.740678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac2c27-f910-49a4-921c-439dff6006cc admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:25.740961 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac2c27-f910-49a4-921c-439dff6006cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:25.741235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac2c27-f910-49a4-921c-439dff6006cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:25.742404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac2c27-f910-49a4-921c-439dff6006cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:25.745808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac2c27-f910-49a4-921c-439dff6006cc admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:25.779087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1112/2240] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:25 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:00:25.782895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc7258-3fcf-4031-9e30-b49f4caa0e55 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:25.795477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc7258-3fcf-4031-9e30-b49f4caa0e55 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:25.859126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe65e67-058a-484c-a147-56d4cbdacc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:25.860858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe65e67-058a-484c-a147-56d4cbdacc8d admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:25.861068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe65e67-058a-484c-a147-56d4cbdacc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:25.861348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe65e67-058a-484c-a147-56d4cbdacc8d admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:25.862344 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe65e67-058a-484c-a147-56d4cbdacc8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:25.864341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe65e67-058a-484c-a147-56d4cbdacc8d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:25.890431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1129/2241] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:25 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:25.894602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac2c27-f910-49a4-921c-439dff6006cc admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:25.907096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac2c27-f910-49a4-921c-439dff6006cc admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:25.912094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fe973-2e3f-4e65-87de-c1f4806cd69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:25.912868 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fe973-2e3f-4e65-87de-c1f4806cd69f admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:25.913209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fe973-2e3f-4e65-87de-c1f4806cd69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:25.913548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fe973-2e3f-4e65-87de-c1f4806cd69f admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:25.914405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fe973-2e3f-4e65-87de-c1f4806cd69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:25.916132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fe973-2e3f-4e65-87de-c1f4806cd69f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:25.929960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1113/2242] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:25.934027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe65e67-058a-484c-a147-56d4cbdacc8d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:25.947266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe65e67-058a-484c-a147-56d4cbdacc8d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:25.953084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa46ef7-cb50-4f1a-a21b-26870d8aa39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:25.953761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa46ef7-cb50-4f1a-a21b-26870d8aa39b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:25.954035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa46ef7-cb50-4f1a-a21b-26870d8aa39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:25.954331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa46ef7-cb50-4f1a-a21b-26870d8aa39b admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:25.955126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa46ef7-cb50-4f1a-a21b-26870d8aa39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:25.956851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa46ef7-cb50-4f1a-a21b-26870d8aa39b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:25.974219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1130/2243] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:25.975532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fe973-2e3f-4e65-87de-c1f4806cd69f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:25.986971 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fe973-2e3f-4e65-87de-c1f4806cd69f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:25.993468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91731d60-d8ba-496a-80c6-ce424100a401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:25.994555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731d60-d8ba-496a-80c6-ce424100a401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:25.994723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731d60-d8ba-496a-80c6-ce424100a401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:25.995162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731d60-d8ba-496a-80c6-ce424100a401 admin admin] PATH_INFO: `/v3/projects/e2351ce4ef984c52aefc7d8cb19ed771/users/62ae7281827a459f906ab3708a4cd261/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:26.012000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91731d60-d8ba-496a-80c6-ce424100a401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2351ce4ef984c52aefc7d8cb19ed771, user_id=62ae7281827a459f906ab3708a4cd261, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:26.014970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91731d60-d8ba-496a-80c6-ce424100a401 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:26.038970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1114/2244] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:25 2022] PUT /identity/v3/projects/e2351ce4ef984c52aefc7d8cb19ed771/users/62ae7281827a459f906ab3708a4cd261/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:26.043515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e5ed-2231-46cf-99fd-b0dfdce83b27 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:26.044131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e5ed-2231-46cf-99fd-b0dfdce83b27 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:26.044632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e5ed-2231-46cf-99fd-b0dfdce83b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:26.080356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-36b6e5ed-2231-46cf-99fd-b0dfdce83b27 None None] MFA Rules not processed for user `62ae7281827a459f906ab3708a4cd261`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:26.081648 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-36b6e5ed-2231-46cf-99fd-b0dfdce83b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:26.225166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1131/2245] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:26.233153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91731d60-d8ba-496a-80c6-ce424100a401 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:26.235302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-91731d60-d8ba-496a-80c6-ce424100a401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:26.278501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:26.322684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91731d60-d8ba-496a-80c6-ce424100a401 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:26.326549 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4676b4d-f487-42f0-8451-a242e5aba3ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:26.327350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676b4d-f487-42f0-8451-a242e5aba3ba service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:26.328002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676b4d-f487-42f0-8451-a242e5aba3ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:26.328208 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676b4d-f487-42f0-8451-a242e5aba3ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:26.338921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4676b4d-f487-42f0-8451-a242e5aba3ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:26.340544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4676b4d-f487-42f0-8451-a242e5aba3ba service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:26.352734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1115/2246] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:00:26.377139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6e5ed-2231-46cf-99fd-b0dfdce83b27 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:26.437817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:26.452376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6e5ed-2231-46cf-99fd-b0dfdce83b27 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:26.457547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:26.458617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:26.458930 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:26.459411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] PATH_INFO: `/v3/users/62ae7281827a459f906ab3708a4cd261` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:26.460749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ae7281827a459f906ab3708a4cd261)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:26.463136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:26.530273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ae7281827a459f906ab3708a4cd261'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:26.545573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ae7281827a459f906ab3708a4cd261'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:26.546217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ae7281827a459f906ab3708a4cd261'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:26.554831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ae7281827a459f906ab3708a4cd261'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:26.555752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ae7281827a459f906ab3708a4cd261'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:26.580000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1132/2247] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:26 2022] DELETE /identity/v3/users/62ae7281827a459f906ab3708a4cd261 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:26.590079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20352657-574d-4c12-ac2c-1af7354d0bc1 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:26.590079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20352657-574d-4c12-ac2c-1af7354d0bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:26.590079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20352657-574d-4c12-ac2c-1af7354d0bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:26.613757 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-20352657-574d-4c12-ac2c-1af7354d0bc1 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:26.616144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-20352657-574d-4c12-ac2c-1af7354d0bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:26.822590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1116/2248] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:00:26.840104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:26.842605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:26.951814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0901e-6ccd-4ae1-9a94-f66e790d57df admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:26.955866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ebe0c-5c2d-439e-bf28-afef455baac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:26.956543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ebe0c-5c2d-439e-bf28-afef455baac3 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:26.956860 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ebe0c-5c2d-439e-bf28-afef455baac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:26.957149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ebe0c-5c2d-439e-bf28-afef455baac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:26.969478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ebe0c-5c2d-439e-bf28-afef455baac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:26.971810 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ebe0c-5c2d-439e-bf28-afef455baac3 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:26.985388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1133/2249] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:27.382424 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20352657-574d-4c12-ac2c-1af7354d0bc1 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:27.384306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-20352657-574d-4c12-ac2c-1af7354d0bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:27.399805 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20352657-574d-4c12-ac2c-1af7354d0bc1 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:27.407721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:27.408855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:27.409341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:27.409904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] PATH_INFO: `/v3/projects/e2351ce4ef984c52aefc7d8cb19ed771` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:27.411229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2351ce4ef984c52aefc7d8cb19ed771)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:27.413239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:27.483931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2351ce4ef984c52aefc7d8cb19ed771', 'request_id': 'req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:27.494007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2351ce4ef984c52aefc7d8cb19ed771', 'request_id': 'req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:27.500572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2351ce4ef984c52aefc7d8cb19ed771', 'request_id': 'req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:27.512946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] The token cache is being invalidate because project e2351ce4ef984c52aefc7d8cb19ed771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:27.513227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:27.513949 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c3c64ac7-acab-45c9-97a1-aaef40ca9eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:27.515626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1117/2250] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:27 2022] DELETE /identity/v3/projects/e2351ce4ef984c52aefc7d8cb19ed771 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:27.520969 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd706b-0783-4ed3-b1e1-77ab906c418d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:27.521513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd706b-0783-4ed3-b1e1-77ab906c418d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:27.521679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd706b-0783-4ed3-b1e1-77ab906c418d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:27.542459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6bbd706b-0783-4ed3-b1e1-77ab906c418d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:27.543686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6bbd706b-0783-4ed3-b1e1-77ab906c418d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:27.628162 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1134/2251] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:27.638311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16070117-b664-40ad-aa50-14c2597a50c3 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:27.638844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16070117-b664-40ad-aa50-14c2597a50c3 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:27.639316 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16070117-b664-40ad-aa50-14c2597a50c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:27.665093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-16070117-b664-40ad-aa50-14c2597a50c3 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:27.666590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-16070117-b664-40ad-aa50-14c2597a50c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:27.677135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1118/2252] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:00:27.683713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd706b-0783-4ed3-b1e1-77ab906c418d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:27.694423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd706b-0783-4ed3-b1e1-77ab906c418d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:27.705357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d671b00-bbc8-45f4-b887-23d923807fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:27.707048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671b00-bbc8-45f4-b887-23d923807fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:27.707537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671b00-bbc8-45f4-b887-23d923807fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:27.708109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671b00-bbc8-45f4-b887-23d923807fd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:27.709438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d671b00-bbc8-45f4-b887-23d923807fd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:00:27.709592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d671b00-bbc8-45f4-b887-23d923807fd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:27.712419 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d671b00-bbc8-45f4-b887-23d923807fd1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:27.742770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1135/2253] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:00:27.744160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16070117-b664-40ad-aa50-14c2597a50c3 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:27.761809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16070117-b664-40ad-aa50-14c2597a50c3 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:27.789134 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9ba72-ac18-477b-bc14-35b0b8b2b54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:27.789920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9ba72-ac18-477b-bc14-35b0b8b2b54b admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:27.790223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9ba72-ac18-477b-bc14-35b0b8b2b54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:27.790502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9ba72-ac18-477b-bc14-35b0b8b2b54b admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:27.791371 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9ba72-ac18-477b-bc14-35b0b8b2b54b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:27.794772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9ba72-ac18-477b-bc14-35b0b8b2b54b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:27.848656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1119/2254] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:27 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:27.853392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d671b00-bbc8-45f4-b887-23d923807fd1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:27.859376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:27.876125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d671b00-bbc8-45f4-b887-23d923807fd1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:28.064112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8ab2c-c7e4-473f-a16a-1c2acdf4851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:28.064841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8ab2c-c7e4-473f-a16a-1c2acdf4851f admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:28.065132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8ab2c-c7e4-473f-a16a-1c2acdf4851f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:28.065517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8ab2c-c7e4-473f-a16a-1c2acdf4851f admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:28.066247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8ab2c-c7e4-473f-a16a-1c2acdf4851f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:28.068435 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8ab2c-c7e4-473f-a16a-1c2acdf4851f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:28.137349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1136/2255] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:27 2022] POST /identity/v3/users => generated 285 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:28.138849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9ba72-ac18-477b-bc14-35b0b8b2b54b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:28.157913 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9ba72-ac18-477b-bc14-35b0b8b2b54b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:28.181441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ac7f3-3785-4aca-9b37-bff239707fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:28.182188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ac7f3-3785-4aca-9b37-bff239707fcd admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:28.182469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ac7f3-3785-4aca-9b37-bff239707fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:28.182778 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ac7f3-3785-4aca-9b37-bff239707fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:28.183519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ac7f3-3785-4aca-9b37-bff239707fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:28.185564 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ac7f3-3785-4aca-9b37-bff239707fcd admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:28.225950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1120/2256] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:28.230695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8ab2c-c7e4-473f-a16a-1c2acdf4851f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:28.249169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8ab2c-c7e4-473f-a16a-1c2acdf4851f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:28.295973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e5fd0-21f5-4e7b-8c15-0737eeb4ae3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:28.326638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e5fd0-21f5-4e7b-8c15-0737eeb4ae3e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:28.326638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e5fd0-21f5-4e7b-8c15-0737eeb4ae3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:28.326638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e5fd0-21f5-4e7b-8c15-0737eeb4ae3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:28.326638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e5fd0-21f5-4e7b-8c15-0737eeb4ae3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:28.326638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e5fd0-21f5-4e7b-8c15-0737eeb4ae3e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:28.357329 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1137/2257] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:28.363122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ac7f3-3785-4aca-9b37-bff239707fcd admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:28.428504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ac7f3-3785-4aca-9b37-bff239707fcd admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:28.441566 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435b2f4-05ff-4059-ac45-b80cf352c6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:28.447228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b2f4-05ff-4059-ac45-b80cf352c6a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:28.447591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b2f4-05ff-4059-ac45-b80cf352c6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:28.448495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b2f4-05ff-4059-ac45-b80cf352c6a3 admin admin] PATH_INFO: `/v3/projects/c58856093213422d855686660348d40a/users/40ba514d7ab34293b62c33f07c6ce975/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:28.491498 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:28.501047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435b2f4-05ff-4059-ac45-b80cf352c6a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58856093213422d855686660348d40a, user_id=40ba514d7ab34293b62c33f07c6ce975, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:28.505278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435b2f4-05ff-4059-ac45-b80cf352c6a3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:28.551968 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1121/2258] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:28 2022] PUT /identity/v3/projects/c58856093213422d855686660348d40a/users/40ba514d7ab34293b62c33f07c6ce975/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:28.562306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6bbd7-506e-49f2-8156-1adfe042d274 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:28.562639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6bbd7-506e-49f2-8156-1adfe042d274 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:28.562901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6bbd7-506e-49f2-8156-1adfe042d274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:28.672785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134f05b-5d69-493e-b5bd-d5533f7ae245 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:28.673408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134f05b-5d69-493e-b5bd-d5533f7ae245 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:28.673408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134f05b-5d69-493e-b5bd-d5533f7ae245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:28.687717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-55e6bbd7-506e-49f2-8156-1adfe042d274 None None] MFA Rules not processed for user `40ba514d7ab34293b62c33f07c6ce975`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:28.689039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-55e6bbd7-506e-49f2-8156-1adfe042d274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:28.701781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-7134f05b-5d69-493e-b5bd-d5533f7ae245 None None] MFA Rules not processed for user `48ff681e601f4892abffe5862def0547`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:28.707305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7134f05b-5d69-493e-b5bd-d5533f7ae245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:28.886650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1122/2259] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:00:28.958771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1138/2260] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:28.973967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134f05b-5d69-493e-b5bd-d5533f7ae245 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:28.980213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7134f05b-5d69-493e-b5bd-d5533f7ae245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:29.149143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134f05b-5d69-493e-b5bd-d5533f7ae245 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:29.157306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f9933-6aac-4d2b-a4ee-468f78714a4b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e850be24df24bcf972e6e31002014cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:29.158271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f9933-6aac-4d2b-a4ee-468f78714a4b service glance] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:29.158688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f9933-6aac-4d2b-a4ee-468f78714a4b service glance] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:29.159074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f9933-6aac-4d2b-a4ee-468f78714a4b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:29.182555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f9933-6aac-4d2b-a4ee-468f78714a4b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:29.184469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f9933-6aac-4d2b-a4ee-468f78714a4b service glance] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:29.206642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1123/2261] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:31.095047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004521f-bf91-4016-8b82-3f96316386c3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:31.095896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004521f-bf91-4016-8b82-3f96316386c3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:31.096441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004521f-bf91-4016-8b82-3f96316386c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:31.125251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7004521f-bf91-4016-8b82-3f96316386c3 None None] MFA Rules not processed for user `229771b215b3492abc2c5ec38ff0d929`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:31.126846 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7004521f-bf91-4016-8b82-3f96316386c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:31.234671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1139/2262] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:39.201361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=54,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:39.384646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f9933-6aac-4d2b-a4ee-468f78714a4b service glance] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:39.386360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-577f9933-6aac-4d2b-a4ee-468f78714a4b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:39.536953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f9933-6aac-4d2b-a4ee-468f78714a4b service glance] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:39.546340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:39.548511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:39.549182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:39.549795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] PATH_INFO: `/v3/users/f78c7829473647929af9a48031814201` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:39.552858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f78c7829473647929af9a48031814201)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:39.557219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:39.626024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f78c7829473647929af9a48031814201'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:39.647775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f78c7829473647929af9a48031814201'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:39.649347 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f78c7829473647929af9a48031814201'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:39.650109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f78c7829473647929af9a48031814201'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:39.665549 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f78c7829473647929af9a48031814201'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:39.687249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1124/2263] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:39 2022] DELETE /identity/v3/users/f78c7829473647929af9a48031814201 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:39.688875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563367-8833-4292-b727-13d26e534cb0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:39.689786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563367-8833-4292-b727-13d26e534cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:39.690234 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563367-8833-4292-b727-13d26e534cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:39.711372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-45563367-8833-4292-b727-13d26e534cb0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:39.713630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-45563367-8833-4292-b727-13d26e534cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:39.726721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:39.825812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1140/2264] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:39.838101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:39.840044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:39.933242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164f277-01c9-4ca0-9f99-cf5f7aded3d7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:39.936990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f901c-16db-4fab-9575-ec8edd380dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:39.937695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f901c-16db-4fab-9575-ec8edd380dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:39.937970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f901c-16db-4fab-9575-ec8edd380dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:39.938236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f901c-16db-4fab-9575-ec8edd380dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:39.952712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f901c-16db-4fab-9575-ec8edd380dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:39.954594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f901c-16db-4fab-9575-ec8edd380dc8 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:39.968698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1125/2265] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:40.406110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45563367-8833-4292-b727-13d26e534cb0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:40.407285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-45563367-8833-4292-b727-13d26e534cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:40.530122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45563367-8833-4292-b727-13d26e534cb0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:40.536285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:40.537227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:40.537506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:40.537781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] PATH_INFO: `/v3/projects/a35c72dcd7fa494c97660b578df6a462` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:40.539323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a35c72dcd7fa494c97660b578df6a462)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:40.541384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:40.616051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a35c72dcd7fa494c97660b578df6a462', 'request_id': 'req-c9b7be1f-e2ff-4c78-87b7-dab065251838'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:40.627395 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a35c72dcd7fa494c97660b578df6a462', 'request_id': 'req-c9b7be1f-e2ff-4c78-87b7-dab065251838'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:40.638989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a35c72dcd7fa494c97660b578df6a462', 'request_id': 'req-c9b7be1f-e2ff-4c78-87b7-dab065251838'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:40.662781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] The token cache is being invalidate because project a35c72dcd7fa494c97660b578df6a462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:40.663085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:40.663816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c9b7be1f-e2ff-4c78-87b7-dab065251838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:40.665520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1141/2266] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:40 2022] DELETE /identity/v3/projects/a35c72dcd7fa494c97660b578df6a462 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:00:40.671064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d94a6-12eb-4658-a815-33631dd2be8b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:40.671980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d94a6-12eb-4658-a815-33631dd2be8b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:40.672443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d94a6-12eb-4658-a815-33631dd2be8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:40.726007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:40.753974 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-4c4d94a6-12eb-4658-a815-33631dd2be8b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:40.755403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4c4d94a6-12eb-4658-a815-33631dd2be8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:40.837471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1126/2267] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:40.846454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0a646-2219-4633-82b4-d550fe234ad9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:40.847091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0a646-2219-4633-82b4-d550fe234ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:40.847204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0a646-2219-4633-82b4-d550fe234ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:40.869039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d7a0a646-2219-4633-82b4-d550fe234ad9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:40.870755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d7a0a646-2219-4633-82b4-d550fe234ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:40.881061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1142/2268] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:40.885844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d94a6-12eb-4658-a815-33631dd2be8b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:40.899551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d94a6-12eb-4658-a815-33631dd2be8b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:40.906398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1984e37-e03f-4f90-b93a-cbdd02823a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:40.908183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1984e37-e03f-4f90-b93a-cbdd02823a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:40.908778 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1984e37-e03f-4f90-b93a-cbdd02823a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:40.909068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1984e37-e03f-4f90-b93a-cbdd02823a2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:40.909920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1984e37-e03f-4f90-b93a-cbdd02823a2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:00:40.910615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1984e37-e03f-4f90-b93a-cbdd02823a2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:40.912244 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1984e37-e03f-4f90-b93a-cbdd02823a2a admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:40.936203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1127/2269] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:00:40.941227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0a646-2219-4633-82b4-d550fe234ad9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:40.953277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0a646-2219-4633-82b4-d550fe234ad9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:40.964438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648afc2d-b072-4c1e-a325-55c5dc00a1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:40.978122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afc2d-b072-4c1e-a325-55c5dc00a1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:40.979044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afc2d-b072-4c1e-a325-55c5dc00a1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:40.979965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afc2d-b072-4c1e-a325-55c5dc00a1b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:40.980911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648afc2d-b072-4c1e-a325-55c5dc00a1b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:40.985806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648afc2d-b072-4c1e-a325-55c5dc00a1b7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:41.025606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1143/2270] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:40 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:41.030221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1984e37-e03f-4f90-b93a-cbdd02823a2a admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:41.052414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1984e37-e03f-4f90-b93a-cbdd02823a2a admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:41.120037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acca3a-2b70-4c91-9e8d-afcd659d8ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:41.120700 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acca3a-2b70-4c91-9e8d-afcd659d8ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:41.120990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acca3a-2b70-4c91-9e8d-afcd659d8ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:41.121263 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acca3a-2b70-4c91-9e8d-afcd659d8ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:41.121973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acca3a-2b70-4c91-9e8d-afcd659d8ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:41.124893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acca3a-2b70-4c91-9e8d-afcd659d8ae6 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:41.151406 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1128/2271] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:41 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:41.156514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648afc2d-b072-4c1e-a325-55c5dc00a1b7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:41.168771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648afc2d-b072-4c1e-a325-55c5dc00a1b7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:41.173668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9544d0-ac08-4ff5-8247-07bff2c38e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:41.174365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9544d0-ac08-4ff5-8247-07bff2c38e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:41.174684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9544d0-ac08-4ff5-8247-07bff2c38e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:41.175053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9544d0-ac08-4ff5-8247-07bff2c38e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:41.175712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9544d0-ac08-4ff5-8247-07bff2c38e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:41.177382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9544d0-ac08-4ff5-8247-07bff2c38e97 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:41.179871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:41.191195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1144/2272] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:41.195800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acca3a-2b70-4c91-9e8d-afcd659d8ae6 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:41.206791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acca3a-2b70-4c91-9e8d-afcd659d8ae6 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:41.212363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f33a7d-9713-41d6-8b40-b9b2036f43e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:41.213105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f33a7d-9713-41d6-8b40-b9b2036f43e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:41.213557 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f33a7d-9713-41d6-8b40-b9b2036f43e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:41.213806 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f33a7d-9713-41d6-8b40-b9b2036f43e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:41.214652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f33a7d-9713-41d6-8b40-b9b2036f43e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:41.216273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f33a7d-9713-41d6-8b40-b9b2036f43e3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:41.228090 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1129/2273] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:41.232406 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9544d0-ac08-4ff5-8247-07bff2c38e97 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:41.243830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9544d0-ac08-4ff5-8247-07bff2c38e97 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:41.248549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dffef0-bd47-4670-8ce2-babd9636c2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:41.249443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dffef0-bd47-4670-8ce2-babd9636c2f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:41.249922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dffef0-bd47-4670-8ce2-babd9636c2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:41.250229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dffef0-bd47-4670-8ce2-babd9636c2f6 admin admin] PATH_INFO: `/v3/projects/ce73b243663d4ee6b991c9262d1286d5/users/f2ed196e5ad6415aa9b7a7c1b3dad277/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:41.273197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dffef0-bd47-4670-8ce2-babd9636c2f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce73b243663d4ee6b991c9262d1286d5, user_id=f2ed196e5ad6415aa9b7a7c1b3dad277, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:41.277273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dffef0-bd47-4670-8ce2-babd9636c2f6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:41.292414 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1145/2274] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:41 2022] PUT /identity/v3/projects/ce73b243663d4ee6b991c9262d1286d5/users/f2ed196e5ad6415aa9b7a7c1b3dad277/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:00:41.297681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef2776-6264-46c3-bcd1-35a09165df1f None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:41.297963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef2776-6264-46c3-bcd1-35a09165df1f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:41.298240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef2776-6264-46c3-bcd1-35a09165df1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:41.318951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-c6ef2776-6264-46c3-bcd1-35a09165df1f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:41.320774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c6ef2776-6264-46c3-bcd1-35a09165df1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:41.391236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1130/2275] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:41.402910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dffef0-bd47-4670-8ce2-babd9636c2f6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:41.404809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-21dffef0-bd47-4670-8ce2-babd9636c2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:41.539059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dffef0-bd47-4670-8ce2-babd9636c2f6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:41.545959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77af55-3123-4ea4-a676-e4c997969dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:41.547021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77af55-3123-4ea4-a676-e4c997969dfe service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:41.547021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77af55-3123-4ea4-a676-e4c997969dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:41.547349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77af55-3123-4ea4-a676-e4c997969dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:41.561444 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77af55-3123-4ea4-a676-e4c997969dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:41.562868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77af55-3123-4ea4-a676-e4c997969dfe service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:41.575621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1146/2276] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:46.301117 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126d390-6a2e-45cc-a50b-0bc267443cc8 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:46.301976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126d390-6a2e-45cc-a50b-0bc267443cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:46.302423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126d390-6a2e-45cc-a50b-0bc267443cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:46.316151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:46.324704 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-7126d390-6a2e-45cc-a50b-0bc267443cc8 None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:46.326326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7126d390-6a2e-45cc-a50b-0bc267443cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:46.412960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1131/2277] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jun 29 10:00:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:46.431097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77af55-3123-4ea4-a676-e4c997969dfe service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:46.433259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0d77af55-3123-4ea4-a676-e4c997969dfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:46.596611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77af55-3123-4ea4-a676-e4c997969dfe service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:46.617873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd5347-3deb-4a4a-b583-fc37270d1942 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:46.618548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd5347-3deb-4a4a-b583-fc37270d1942 service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:46.618924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd5347-3deb-4a4a-b583-fc37270d1942 service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:46.619199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd5347-3deb-4a4a-b583-fc37270d1942 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:46.637356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd5347-3deb-4a4a-b583-fc37270d1942 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:46.640881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd5347-3deb-4a4a-b583-fc37270d1942 service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:46.661094 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1147/2278] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:00:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:00:46.689816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5552831-d349-4f7a-a67a-00a81322fbe2 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:46.690394 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5552831-d349-4f7a-a67a-00a81322fbe2 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:46.690935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5552831-d349-4f7a-a67a-00a81322fbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:46.737745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-f5552831-d349-4f7a-a67a-00a81322fbe2 None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:46.739910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f5552831-d349-4f7a-a67a-00a81322fbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:46.762648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1132/2279] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 10:00:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:46.794166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd5347-3deb-4a4a-b583-fc37270d1942 service swift] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:46.809625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd5347-3deb-4a4a-b583-fc37270d1942 service swift] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:46.829005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd2d8a-900d-4d03-a484-d95b849a5b6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:46.829978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd2d8a-900d-4d03-a484-d95b849a5b6c service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:46.830487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd2d8a-900d-4d03-a484-d95b849a5b6c service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:46.831133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd2d8a-900d-4d03-a484-d95b849a5b6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:46.847307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd2d8a-900d-4d03-a484-d95b849a5b6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:46.849188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd2d8a-900d-4d03-a484-d95b849a5b6c service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:46.870999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1148/2280] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:00:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:00:49.035839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75924c7-a003-43d0-8633-281c2bbadbef None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:49.036811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75924c7-a003-43d0-8633-281c2bbadbef None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:49.037410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75924c7-a003-43d0-8633-281c2bbadbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:49.124670 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b75924c7-a003-43d0-8633-281c2bbadbef None None] MFA Rules not processed for user `f2ed196e5ad6415aa9b7a7c1b3dad277`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:49.126118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b75924c7-a003-43d0-8633-281c2bbadbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:49.277627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1133/2281] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:49.282579 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd2d8a-900d-4d03-a484-d95b849a5b6c service swift] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:49.298558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd2d8a-900d-4d03-a484-d95b849a5b6c service swift] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:49.305441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4ef22-720d-4b39-9e78-6ebace5c2895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:49.306118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4ef22-720d-4b39-9e78-6ebace5c2895 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:49.306394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4ef22-720d-4b39-9e78-6ebace5c2895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:49.306765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4ef22-720d-4b39-9e78-6ebace5c2895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:49.307470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4ef22-720d-4b39-9e78-6ebace5c2895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:49.310008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4ef22-720d-4b39-9e78-6ebace5c2895 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:49.346242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:49.375193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75924c7-a003-43d0-8633-281c2bbadbef None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:49.376392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1149/2282] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:49 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:49.395950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75924c7-a003-43d0-8633-281c2bbadbef None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:49.500474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6d46c-ba53-4e68-b129-00d17f4abe1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:49.501199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6d46c-ba53-4e68-b129-00d17f4abe1a admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:49.501482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6d46c-ba53-4e68-b129-00d17f4abe1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:49.501761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6d46c-ba53-4e68-b129-00d17f4abe1a admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:49.503667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6d46c-ba53-4e68-b129-00d17f4abe1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:49.507521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6d46c-ba53-4e68-b129-00d17f4abe1a admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:49.548234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1134/2283] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:49 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:49.552295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4ef22-720d-4b39-9e78-6ebace5c2895 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:49.565638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4ef22-720d-4b39-9e78-6ebace5c2895 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:49.570913 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab754-14ce-46a9-9ea8-a92ce0a75f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:49.572069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab754-14ce-46a9-9ea8-a92ce0a75f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:49.572454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab754-14ce-46a9-9ea8-a92ce0a75f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:49.572938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab754-14ce-46a9-9ea8-a92ce0a75f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:49.573729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eab754-14ce-46a9-9ea8-a92ce0a75f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:49.575834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eab754-14ce-46a9-9ea8-a92ce0a75f3d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:49.607575 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1150/2284] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:49.607769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6d46c-ba53-4e68-b129-00d17f4abe1a admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:49.620728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6d46c-ba53-4e68-b129-00d17f4abe1a admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:49.632270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8e7ca-8dc7-4a1f-9125-137dc0bfad77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:49.635721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8e7ca-8dc7-4a1f-9125-137dc0bfad77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:49.636118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8e7ca-8dc7-4a1f-9125-137dc0bfad77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:49.636756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8e7ca-8dc7-4a1f-9125-137dc0bfad77 admin admin] PATH_INFO: `/v3/projects/caccf28b6b80494393ac62c201fce18e/users/ea0a851d0a1844e2b9f2f949898b7190/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:49.661631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8e7ca-8dc7-4a1f-9125-137dc0bfad77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caccf28b6b80494393ac62c201fce18e, user_id=ea0a851d0a1844e2b9f2f949898b7190, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:49.664663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8e7ca-8dc7-4a1f-9125-137dc0bfad77 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:49.688479 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:49.696710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1135/2285] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:49 2022] PUT /identity/v3/projects/caccf28b6b80494393ac62c201fce18e/users/ea0a851d0a1844e2b9f2f949898b7190/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:49.700486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab754-14ce-46a9-9ea8-a92ce0a75f3d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:49.725658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab754-14ce-46a9-9ea8-a92ce0a75f3d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:49.880360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4154f7-0664-485c-aac2-5164e3a4321e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:49.881317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4154f7-0664-485c-aac2-5164e3a4321e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:49.884692 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4154f7-0664-485c-aac2-5164e3a4321e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:49.885267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4154f7-0664-485c-aac2-5164e3a4321e admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:49.886051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4154f7-0664-485c-aac2-5164e3a4321e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:49.887748 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4154f7-0664-485c-aac2-5164e3a4321e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:49.921026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1151/2286] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:49.921243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8e7ca-8dc7-4a1f-9125-137dc0bfad77 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:49.940682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8e7ca-8dc7-4a1f-9125-137dc0bfad77 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:49.956557 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273c438-ad4c-4418-96e1-45f9cd9e1625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:49.957592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273c438-ad4c-4418-96e1-45f9cd9e1625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:49.957969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273c438-ad4c-4418-96e1-45f9cd9e1625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:49.958340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273c438-ad4c-4418-96e1-45f9cd9e1625 admin admin] PATH_INFO: `/v3/projects/caccf28b6b80494393ac62c201fce18e/users/ea0a851d0a1844e2b9f2f949898b7190/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:49.962010 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273c438-ad4c-4418-96e1-45f9cd9e1625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caccf28b6b80494393ac62c201fce18e, user_id=ea0a851d0a1844e2b9f2f949898b7190, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:49.965814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273c438-ad4c-4418-96e1-45f9cd9e1625 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:49.995855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1136/2287] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:49 2022] PUT /identity/v3/projects/caccf28b6b80494393ac62c201fce18e/users/ea0a851d0a1844e2b9f2f949898b7190/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:00:50.730616 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4154f7-0664-485c-aac2-5164e3a4321e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:50.733056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8e4154f7-0664-485c-aac2-5164e3a4321e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:50.759672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4154f7-0664-485c-aac2-5164e3a4321e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:50.979462 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bac9a-d546-498d-9c3a-709ee8544bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:50.981638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bac9a-d546-498d-9c3a-709ee8544bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:50.986644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bac9a-d546-498d-9c3a-709ee8544bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:50.988526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bac9a-d546-498d-9c3a-709ee8544bd0 admin admin] PATH_INFO: `/v3/users/1694bd50d03f4d0f9c2ccee1b0775e12` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:50.993221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bac9a-d546-498d-9c3a-709ee8544bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1694bd50d03f4d0f9c2ccee1b0775e12)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:50.997370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bac9a-d546-498d-9c3a-709ee8544bd0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:51.118973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-723bac9a-d546-498d-9c3a-709ee8544bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1694bd50d03f4d0f9c2ccee1b0775e12'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:51.150048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-723bac9a-d546-498d-9c3a-709ee8544bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1694bd50d03f4d0f9c2ccee1b0775e12'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:51.150799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-723bac9a-d546-498d-9c3a-709ee8544bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1694bd50d03f4d0f9c2ccee1b0775e12'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:51.166601 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-723bac9a-d546-498d-9c3a-709ee8544bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1694bd50d03f4d0f9c2ccee1b0775e12'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:51.167417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-723bac9a-d546-498d-9c3a-709ee8544bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1694bd50d03f4d0f9c2ccee1b0775e12'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:51.183893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:51.193819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1152/2288] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:00:50 2022] DELETE /identity/v3/users/1694bd50d03f4d0f9c2ccee1b0775e12 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:51.983761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273c438-ad4c-4418-96e1-45f9cd9e1625 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:51.986479 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e273c438-ad4c-4418-96e1-45f9cd9e1625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:52.006173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273c438-ad4c-4418-96e1-45f9cd9e1625 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:52.277799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:52.278731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:52.279038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:52.279273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] PATH_INFO: `/v3/projects/5c61e1258f5d44ba903a0132415437c6` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:52.284605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c61e1258f5d44ba903a0132415437c6)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:52.286624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:52.447642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c61e1258f5d44ba903a0132415437c6', 'request_id': 'req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:52.473111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c61e1258f5d44ba903a0132415437c6', 'request_id': 'req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:52.489763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c61e1258f5d44ba903a0132415437c6', 'request_id': 'req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:52.528722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] The token cache is being invalidate because project 5c61e1258f5d44ba903a0132415437c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:00:52.529018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:52.531702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-1d6a36df-eaad-4ee7-bfab-a7b92f52106d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:00:52.537159 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1137/2289] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:00:51 2022] DELETE /identity/v3/projects/5c61e1258f5d44ba903a0132415437c6 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:00:52.542381 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62ca3-36b8-4aa7-86c1-e54f1d5b5043 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:52.543514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62ca3-36b8-4aa7-86c1-e54f1d5b5043 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:52.543988 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62ca3-36b8-4aa7-86c1-e54f1d5b5043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:52.588863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e4f62ca3-36b8-4aa7-86c1-e54f1d5b5043 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:52.591671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e4f62ca3-36b8-4aa7-86c1-e54f1d5b5043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:52.836114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1153/2290] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:52.837145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68eb36-d4cd-42ab-a7aa-1fdc4def81da None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:52.837602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68eb36-d4cd-42ab-a7aa-1fdc4def81da None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:52.838073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68eb36-d4cd-42ab-a7aa-1fdc4def81da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:52.912955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-1c68eb36-d4cd-42ab-a7aa-1fdc4def81da None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:52.914404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-1c68eb36-d4cd-42ab-a7aa-1fdc4def81da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:52.940520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1138/2291] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:52.945038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f62ca3-36b8-4aa7-86c1-e54f1d5b5043 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:52.969568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f62ca3-36b8-4aa7-86c1-e54f1d5b5043 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:52.983995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92184d91-bb0b-4bc0-a9e1-7cbc3056e24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:52.985099 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92184d91-bb0b-4bc0-a9e1-7cbc3056e24c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:52.985099 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92184d91-bb0b-4bc0-a9e1-7cbc3056e24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:52.985391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92184d91-bb0b-4bc0-a9e1-7cbc3056e24c admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:52.988224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92184d91-bb0b-4bc0-a9e1-7cbc3056e24c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:00:52.988944 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92184d91-bb0b-4bc0-a9e1-7cbc3056e24c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:52.990750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92184d91-bb0b-4bc0-a9e1-7cbc3056e24c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:53.050833 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1154/2292] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:00:53.055472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68eb36-d4cd-42ab-a7aa-1fdc4def81da None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:53.070181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68eb36-d4cd-42ab-a7aa-1fdc4def81da None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:53.097562 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089c02b-bef7-4c7f-bf83-a924b6c7796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:53.098645 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089c02b-bef7-4c7f-bf83-a924b6c7796b admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:53.099147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089c02b-bef7-4c7f-bf83-a924b6c7796b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:53.099743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089c02b-bef7-4c7f-bf83-a924b6c7796b admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:53.105650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089c02b-bef7-4c7f-bf83-a924b6c7796b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:53.117303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089c02b-bef7-4c7f-bf83-a924b6c7796b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:53.215033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92184d91-bb0b-4bc0-a9e1-7cbc3056e24c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:53.223495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1139/2293] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:00:53 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:53.234532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92184d91-bb0b-4bc0-a9e1-7cbc3056e24c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:53.439607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d2989-b1d1-4fa4-9aaf-e58ae1c70206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:53.442092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d2989-b1d1-4fa4-9aaf-e58ae1c70206 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:53.442845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d2989-b1d1-4fa4-9aaf-e58ae1c70206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:53.442845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d2989-b1d1-4fa4-9aaf-e58ae1c70206 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:53.443631 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d2989-b1d1-4fa4-9aaf-e58ae1c70206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:53.450427 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d2989-b1d1-4fa4-9aaf-e58ae1c70206 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:53.498299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089c02b-bef7-4c7f-bf83-a924b6c7796b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:53.505394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1155/2294] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:00:53 2022] POST /identity/v3/users => generated 292 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:00:53.515284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089c02b-bef7-4c7f-bf83-a924b6c7796b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:53.547290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a4c01-4bf2-4e63-8329-08c4c20a3143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:53.548279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a4c01-4bf2-4e63-8329-08c4c20a3143 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:53.548562 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a4c01-4bf2-4e63-8329-08c4c20a3143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:53.548826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a4c01-4bf2-4e63-8329-08c4c20a3143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:53.549578 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a4c01-4bf2-4e63-8329-08c4c20a3143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:53.555493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a4c01-4bf2-4e63-8329-08c4c20a3143 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:53.580504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1140/2295] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 10:00:53.584590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d2989-b1d1-4fa4-9aaf-e58ae1c70206 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:53.595987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d2989-b1d1-4fa4-9aaf-e58ae1c70206 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:53.618038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b508f64-b35c-4a9b-b6a2-7b627f26511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:53.619111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b508f64-b35c-4a9b-b6a2-7b627f26511f admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:53.619111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b508f64-b35c-4a9b-b6a2-7b627f26511f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:53.619283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b508f64-b35c-4a9b-b6a2-7b627f26511f admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:53.620018 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b508f64-b35c-4a9b-b6a2-7b627f26511f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:53.621804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b508f64-b35c-4a9b-b6a2-7b627f26511f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:53.655888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a4c01-4bf2-4e63-8329-08c4c20a3143 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:53.657560 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1156/2296] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:00:53.668812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a4c01-4bf2-4e63-8329-08c4c20a3143 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:53.678731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8483c50-7be9-403e-95c7-27bd43cc7c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:53.679687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8483c50-7be9-403e-95c7-27bd43cc7c2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:53.679965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8483c50-7be9-403e-95c7-27bd43cc7c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:53.680236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8483c50-7be9-403e-95c7-27bd43cc7c2d admin admin] PATH_INFO: `/v3/projects/a6f311c41ad04603bf1037c9c3733808/users/c34964aa209f4f48a3f9fbc3e534a589/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:53.721995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8483c50-7be9-403e-95c7-27bd43cc7c2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f311c41ad04603bf1037c9c3733808, user_id=c34964aa209f4f48a3f9fbc3e534a589, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:53.733944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8483c50-7be9-403e-95c7-27bd43cc7c2d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:53.783246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1141/2297] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:00:53 2022] PUT /identity/v3/projects/a6f311c41ad04603bf1037c9c3733808/users/c34964aa209f4f48a3f9fbc3e534a589/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:00:53.794710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0c92a-36b2-473b-b5b2-0265d8bd36fc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:53.794996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0c92a-36b2-473b-b5b2-0265d8bd36fc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:53.795275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0c92a-36b2-473b-b5b2-0265d8bd36fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:53.873196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dec0c92a-36b2-473b-b5b2-0265d8bd36fc None None] MFA Rules not processed for user `c34964aa209f4f48a3f9fbc3e534a589`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:00:53.876415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dec0c92a-36b2-473b-b5b2-0265d8bd36fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:54.035596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:54.078046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8483c50-7be9-403e-95c7-27bd43cc7c2d admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:54.079633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1157/2298] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:00:54.080697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e8483c50-7be9-403e-95c7-27bd43cc7c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:54.115193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:00:54.263871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8483c50-7be9-403e-95c7-27bd43cc7c2d admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:54.278322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23bb5a-2ccb-457e-8a7f-fcd95e9e2d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:54.279746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23bb5a-2ccb-457e-8a7f-fcd95e9e2d8e service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:54.280273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23bb5a-2ccb-457e-8a7f-fcd95e9e2d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:54.280750 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23bb5a-2ccb-457e-8a7f-fcd95e9e2d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:54.295682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23bb5a-2ccb-457e-8a7f-fcd95e9e2d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:54.319900 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23bb5a-2ccb-457e-8a7f-fcd95e9e2d8e service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:54.363791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1142/2299] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:00:55.497769 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0c92a-36b2-473b-b5b2-0265d8bd36fc None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:00:55.500090 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dec0c92a-36b2-473b-b5b2-0265d8bd36fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:55.605712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0c92a-36b2-473b-b5b2-0265d8bd36fc None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:00:55.611378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e14210-206a-4d0d-944d-02a02735aa95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:00:55.612277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e14210-206a-4d0d-944d-02a02735aa95 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:00:55.612707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e14210-206a-4d0d-944d-02a02735aa95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:00:55.613126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e14210-206a-4d0d-944d-02a02735aa95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:00:55.616094 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e9e14210-206a-4d0d-944d-02a02735aa95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:00:55.714867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e14210-206a-4d0d-944d-02a02735aa95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:00:55.716644 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e14210-206a-4d0d-944d-02a02735aa95 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:00:55.738894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1158/2300] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:00:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:00.168530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23bb5a-2ccb-457e-8a7f-fcd95e9e2d8e service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:00.360419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23bb5a-2ccb-457e-8a7f-fcd95e9e2d8e service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:00.367473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:00.368388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:00.368672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:00.369540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] PATH_INFO: `/v3/users/c34964aa209f4f48a3f9fbc3e534a589` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:00.370920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34964aa209f4f48a3f9fbc3e534a589)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:00.380619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:00.528827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34964aa209f4f48a3f9fbc3e534a589'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:00.563594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34964aa209f4f48a3f9fbc3e534a589'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:00.564279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34964aa209f4f48a3f9fbc3e534a589'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:00.564905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34964aa209f4f48a3f9fbc3e534a589'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:00.593996 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34964aa209f4f48a3f9fbc3e534a589'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:00.661622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1143/2301] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:01:00 2022] DELETE /identity/v3/users/c34964aa209f4f48a3f9fbc3e534a589 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:01:00.665842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b5e5e-16c0-4496-b96a-35f4679c119b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:00.666401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b5e5e-16c0-4496-b96a-35f4679c119b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:00.666683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b5e5e-16c0-4496-b96a-35f4679c119b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:00.716208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-064b5e5e-16c0-4496-b96a-35f4679c119b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:00.717867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-064b5e5e-16c0-4496-b96a-35f4679c119b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:00.856011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1159/2302] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:00.874057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:00.876616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:00.955111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:01.017763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab21ff-8bfc-40bc-ad1c-c1134269eed4 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:01.026756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af34cde-590f-4880-82c7-2607ca801b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:01.029611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af34cde-590f-4880-82c7-2607ca801b3e service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:01.030932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af34cde-590f-4880-82c7-2607ca801b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:01.031317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af34cde-590f-4880-82c7-2607ca801b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:01.087969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af34cde-590f-4880-82c7-2607ca801b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:01.092757 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af34cde-590f-4880-82c7-2607ca801b3e service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:01.123591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1144/2303] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:01.183305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b398b8-8543-4768-8a61-054ee7ce43bf None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:01.188756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b398b8-8543-4768-8a61-054ee7ce43bf None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:01.189668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b398b8-8543-4768-8a61-054ee7ce43bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:01.287990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-55b398b8-8543-4768-8a61-054ee7ce43bf None None] MFA Rules not processed for user `ea0a851d0a1844e2b9f2f949898b7190`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:01.317086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-55b398b8-8543-4768-8a61-054ee7ce43bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:01.450851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:01.541331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1160/2304] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:01:01.555570 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af34cde-590f-4880-82c7-2607ca801b3e service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:01.557323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6af34cde-590f-4880-82c7-2607ca801b3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:01.739232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af34cde-590f-4880-82c7-2607ca801b3e service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:01.748104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbc953-09e5-43dd-8ed3-41db29d93740 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e850be24df24bcf972e6e31002014cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:01.749129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbc953-09e5-43dd-8ed3-41db29d93740 service glance] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:01.749428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbc953-09e5-43dd-8ed3-41db29d93740 service glance] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:01.749706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbc953-09e5-43dd-8ed3-41db29d93740 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:01.752416 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-48cbc953-09e5-43dd-8ed3-41db29d93740 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:01.883520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbc953-09e5-43dd-8ed3-41db29d93740 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:01.885425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbc953-09e5-43dd-8ed3-41db29d93740 service glance] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:01.900069 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1145/2305] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:02.181340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e0ebc-4696-4670-8b70-8e640bcce6ee None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:02.182459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e0ebc-4696-4670-8b70-8e640bcce6ee None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:02.182910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e0ebc-4696-4670-8b70-8e640bcce6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:02.238847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-553e0ebc-4696-4670-8b70-8e640bcce6ee None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:02.244646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-553e0ebc-4696-4670-8b70-8e640bcce6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:02.255945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbc953-09e5-43dd-8ed3-41db29d93740 service glance] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:02.258075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-48cbc953-09e5-43dd-8ed3-41db29d93740 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:02.287223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbc953-09e5-43dd-8ed3-41db29d93740 service glance] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:02.300086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:02.300994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:02.301270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:02.301541 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] PATH_INFO: `/v3/projects/a6f311c41ad04603bf1037c9c3733808` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:02.305918 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f311c41ad04603bf1037c9c3733808)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:02.310369 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:02.416108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1161/2306] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 29 10:01:02.423629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e0ebc-4696-4670-8b70-8e640bcce6ee None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:02.426426 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-553e0ebc-4696-4670-8b70-8e640bcce6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:02.454948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f311c41ad04603bf1037c9c3733808', 'request_id': 'req-f46cda1f-1f4d-4120-b28c-572fe4a983a9'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:02.476257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f311c41ad04603bf1037c9c3733808', 'request_id': 'req-f46cda1f-1f4d-4120-b28c-572fe4a983a9'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:02.509332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f311c41ad04603bf1037c9c3733808', 'request_id': 'req-f46cda1f-1f4d-4120-b28c-572fe4a983a9'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:02.547638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e0ebc-4696-4670-8b70-8e640bcce6ee None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:02.548474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] The token cache is being invalidate because project a6f311c41ad04603bf1037c9c3733808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:01:02.549271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:02.550124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f46cda1f-1f4d-4120-b28c-572fe4a983a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:02.553878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1146/2307] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:01:02 2022] DELETE /identity/v3/projects/a6f311c41ad04603bf1037c9c3733808 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:01:02.559197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd474e8f-e80c-4fa5-ba4d-5124a627136c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:02.559765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd474e8f-e80c-4fa5-ba4d-5124a627136c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:02.559765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd474e8f-e80c-4fa5-ba4d-5124a627136c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:02.664302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:02.689709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fd474e8f-e80c-4fa5-ba4d-5124a627136c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:02.691165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fd474e8f-e80c-4fa5-ba4d-5124a627136c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:02.808673 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe5fdd-365d-45d6-9abd-61b71fa1dab9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:02.809367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe5fdd-365d-45d6-9abd-61b71fa1dab9 service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:02.809763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe5fdd-365d-45d6-9abd-61b71fa1dab9 service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:02.810053 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe5fdd-365d-45d6-9abd-61b71fa1dab9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:02.817650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-72fe5fdd-365d-45d6-9abd-61b71fa1dab9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:02.847297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:02.867353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1147/2308] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:02.875772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b9cb1-f8a1-4043-80a0-70a84e42b2dd None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:02.876320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b9cb1-f8a1-4043-80a0-70a84e42b2dd None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:02.876764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b9cb1-f8a1-4043-80a0-70a84e42b2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:02.941362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-683b9cb1-f8a1-4043-80a0-70a84e42b2dd None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:02.952637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-683b9cb1-f8a1-4043-80a0-70a84e42b2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:02.981968 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1148/2309] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:02.987345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b9cb1-f8a1-4043-80a0-70a84e42b2dd None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:03.018096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe5fdd-365d-45d6-9abd-61b71fa1dab9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:03.019781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe5fdd-365d-45d6-9abd-61b71fa1dab9 service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:03.027572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b9cb1-f8a1-4043-80a0-70a84e42b2dd None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:03.038225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7e936-b10a-4f1e-80ab-9d5dea7ac57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:03.048814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7e936-b10a-4f1e-80ab-9d5dea7ac57f admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:03.058717 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7e936-b10a-4f1e-80ab-9d5dea7ac57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:03.059188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7e936-b10a-4f1e-80ab-9d5dea7ac57f admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:03.060685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7e936-b10a-4f1e-80ab-9d5dea7ac57f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:01:03.061255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7e936-b10a-4f1e-80ab-9d5dea7ac57f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:03.062636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1162/2310] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:01:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:01:03.063656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7e936-b10a-4f1e-80ab-9d5dea7ac57f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:03.093616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1149/2311] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:01:03.112145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe5fdd-365d-45d6-9abd-61b71fa1dab9 service swift] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:03.158861 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe5fdd-365d-45d6-9abd-61b71fa1dab9 service swift] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:03.168804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abb7f1-71f8-4c8a-90f7-4fe116513c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:03.170093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abb7f1-71f8-4c8a-90f7-4fe116513c81 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:03.170651 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abb7f1-71f8-4c8a-90f7-4fe116513c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:03.171164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abb7f1-71f8-4c8a-90f7-4fe116513c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:03.172228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abb7f1-71f8-4c8a-90f7-4fe116513c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:03.175788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abb7f1-71f8-4c8a-90f7-4fe116513c81 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:03.247093 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1163/2312] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:01:03 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:01:03.248473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7e936-b10a-4f1e-80ab-9d5dea7ac57f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:03.260163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7e936-b10a-4f1e-80ab-9d5dea7ac57f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:03.451945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54a3b5-69a3-4428-a417-07dd0cb29572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:03.455028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54a3b5-69a3-4428-a417-07dd0cb29572 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:03.455551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54a3b5-69a3-4428-a417-07dd0cb29572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:03.455676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54a3b5-69a3-4428-a417-07dd0cb29572 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:03.457092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54a3b5-69a3-4428-a417-07dd0cb29572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:03.459061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54a3b5-69a3-4428-a417-07dd0cb29572 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:03.574945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1150/2313] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:01:03 2022] POST /identity/v3/users => generated 283 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:01:03.577267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abb7f1-71f8-4c8a-90f7-4fe116513c81 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:03.593388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abb7f1-71f8-4c8a-90f7-4fe116513c81 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:03.625862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34489a0b-dde9-429a-8001-0c52b86885bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:03.626694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34489a0b-dde9-429a-8001-0c52b86885bf admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:03.627043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34489a0b-dde9-429a-8001-0c52b86885bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:03.627391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34489a0b-dde9-429a-8001-0c52b86885bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:03.628213 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34489a0b-dde9-429a-8001-0c52b86885bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:03.629941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34489a0b-dde9-429a-8001-0c52b86885bf admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:03.658499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1164/2314] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:01:03.662301 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54a3b5-69a3-4428-a417-07dd0cb29572 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:03.684720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54a3b5-69a3-4428-a417-07dd0cb29572 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:03.704397 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9b838-e6ad-495f-b80b-249d172c4cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:03.705111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9b838-e6ad-495f-b80b-249d172c4cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:03.705399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9b838-e6ad-495f-b80b-249d172c4cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:03.705681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9b838-e6ad-495f-b80b-249d172c4cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:03.706439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9b838-e6ad-495f-b80b-249d172c4cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:03.714492 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9b838-e6ad-495f-b80b-249d172c4cb9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:03.742401 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1151/2315] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:01:03.746765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34489a0b-dde9-429a-8001-0c52b86885bf admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:03.764671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34489a0b-dde9-429a-8001-0c52b86885bf admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:03.789471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea856c8e-f555-4198-bdfb-c8424c517558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:03.790679 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea856c8e-f555-4198-bdfb-c8424c517558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:03.791146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea856c8e-f555-4198-bdfb-c8424c517558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:03.791587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea856c8e-f555-4198-bdfb-c8424c517558 admin admin] PATH_INFO: `/v3/projects/442f757fbece4bf8a0021dd4047fa468/users/6dbb8c0ccb274fbfb460d0afda35802c/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:03.847459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea856c8e-f555-4198-bdfb-c8424c517558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=442f757fbece4bf8a0021dd4047fa468, user_id=6dbb8c0ccb274fbfb460d0afda35802c, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:03.850512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea856c8e-f555-4198-bdfb-c8424c517558 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:03.881772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1165/2316] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:01:03 2022] PUT /identity/v3/projects/442f757fbece4bf8a0021dd4047fa468/users/6dbb8c0ccb274fbfb460d0afda35802c/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:01:03.892752 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a1141-5f92-4107-a80a-febb605dc918 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:03.893045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a1141-5f92-4107-a80a-febb605dc918 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:03.893327 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a1141-5f92-4107-a80a-febb605dc918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:03.936000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-581a1141-5f92-4107-a80a-febb605dc918 None None] MFA Rules not processed for user `6dbb8c0ccb274fbfb460d0afda35802c`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:03.937364 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-581a1141-5f92-4107-a80a-febb605dc918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:04.022288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:04.049711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1152/2317] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:04.062476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea856c8e-f555-4198-bdfb-c8424c517558 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:04.064485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ea856c8e-f555-4198-bdfb-c8424c517558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:04.222227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea856c8e-f555-4198-bdfb-c8424c517558 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:04.232711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5b5e4-2610-42ae-a2eb-5cb687c4b0be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:04.233545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5b5e4-2610-42ae-a2eb-5cb687c4b0be service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:04.233927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5b5e4-2610-42ae-a2eb-5cb687c4b0be service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:04.234302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5b5e4-2610-42ae-a2eb-5cb687c4b0be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:04.255302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5b5e4-2610-42ae-a2eb-5cb687c4b0be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:04.257512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5b5e4-2610-42ae-a2eb-5cb687c4b0be service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:04.290727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1166/2318] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:05.197331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a1141-5f92-4107-a80a-febb605dc918 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:05.199501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-581a1141-5f92-4107-a80a-febb605dc918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:05.407336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a1141-5f92-4107-a80a-febb605dc918 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:05.426895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:05.428579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:05.429016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:05.429389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] PATH_INFO: `/v3/users/40ba514d7ab34293b62c33f07c6ce975` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:05.432140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40ba514d7ab34293b62c33f07c6ce975)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:05.434418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:05.559459 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40ba514d7ab34293b62c33f07c6ce975'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:05.589017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40ba514d7ab34293b62c33f07c6ce975'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:05.591068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40ba514d7ab34293b62c33f07c6ce975'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:05.593450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40ba514d7ab34293b62c33f07c6ce975'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:05.610725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40ba514d7ab34293b62c33f07c6ce975'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:05.646857 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1153/2319] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:01:05 2022] DELETE /identity/v3/users/40ba514d7ab34293b62c33f07c6ce975 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:01:05.649083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab8f1-085a-4fc5-bf07-48a5cdbfa0bf None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:05.650104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab8f1-085a-4fc5-bf07-48a5cdbfa0bf None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:05.650358 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab8f1-085a-4fc5-bf07-48a5cdbfa0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:05.687126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2deab8f1-085a-4fc5-bf07-48a5cdbfa0bf None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:05.690033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2deab8f1-085a-4fc5-bf07-48a5cdbfa0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:05.832060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:05.863130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1167/2320] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:05.871061 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:05.872855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:06.061260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6a0b3-7eaa-4e23-840c-0a55e1eb7a45 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:06.082770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e9b6e-cc10-49b7-9a40-05bccf1f0e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:06.083643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e9b6e-cc10-49b7-9a40-05bccf1f0e36 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:06.084099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e9b6e-cc10-49b7-9a40-05bccf1f0e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:06.084530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e9b6e-cc10-49b7-9a40-05bccf1f0e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:06.112929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e9b6e-cc10-49b7-9a40-05bccf1f0e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:06.116147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e9b6e-cc10-49b7-9a40-05bccf1f0e36 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:06.172381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1154/2321] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:07.312344 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deab8f1-085a-4fc5-bf07-48a5cdbfa0bf None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:07.313850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2deab8f1-085a-4fc5-bf07-48a5cdbfa0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:07.346244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deab8f1-085a-4fc5-bf07-48a5cdbfa0bf None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:07.384516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:07.390750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:07.394023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:07.397821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] PATH_INFO: `/v3/projects/c58856093213422d855686660348d40a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:07.415826 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58856093213422d855686660348d40a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:07.417072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:07.533077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58856093213422d855686660348d40a', 'request_id': 'req-5db64b2e-e861-47b1-9f6d-a0a4a5624692'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:07.555615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58856093213422d855686660348d40a', 'request_id': 'req-5db64b2e-e861-47b1-9f6d-a0a4a5624692'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:07.563954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58856093213422d855686660348d40a', 'request_id': 'req-5db64b2e-e861-47b1-9f6d-a0a4a5624692'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:07.595894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] The token cache is being invalidate because project c58856093213422d855686660348d40a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:01:07.596155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:07.596843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5db64b2e-e861-47b1-9f6d-a0a4a5624692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:07.604091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1168/2322] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:01:07 2022] DELETE /identity/v3/projects/c58856093213422d855686660348d40a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:01:07.605355 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336f031-4209-4503-a22f-c1d5d725c883 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:07.605872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336f031-4209-4503-a22f-c1d5d725c883 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:07.606174 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336f031-4209-4503-a22f-c1d5d725c883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:07.620627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246c142-7174-4e6c-a37d-d25009af428a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:07.620923 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246c142-7174-4e6c-a37d-d25009af428a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:07.621206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246c142-7174-4e6c-a37d-d25009af428a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:07.652887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a336f031-4209-4503-a22f-c1d5d725c883 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:07.654240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a336f031-4209-4503-a22f-c1d5d725c883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:07.671376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:07.686870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f246c142-7174-4e6c-a37d-d25009af428a None None] MFA Rules not processed for user `63ba0760d13e4bbda0650de803ef46ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:07.693607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f246c142-7174-4e6c-a37d-d25009af428a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:07.776392 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1155/2323] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:01:07.785965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15beed-7d41-44ba-8a25-293ff1030a83 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:07.786298 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15beed-7d41-44ba-8a25-293ff1030a83 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:07.786527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15beed-7d41-44ba-8a25-293ff1030a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:07.827745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6d15beed-7d41-44ba-8a25-293ff1030a83 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:07.828993 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6d15beed-7d41-44ba-8a25-293ff1030a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:07.853643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1156/2324] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:07.859831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15beed-7d41-44ba-8a25-293ff1030a83 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:07.897983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15beed-7d41-44ba-8a25-293ff1030a83 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:07.906647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1169/2325] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:07.922719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796d37f-0b1f-4ac8-8c62-090c8691638d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:07.923517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796d37f-0b1f-4ac8-8c62-090c8691638d admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:07.923868 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796d37f-0b1f-4ac8-8c62-090c8691638d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:07.924307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796d37f-0b1f-4ac8-8c62-090c8691638d admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:07.925148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796d37f-0b1f-4ac8-8c62-090c8691638d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:01:07.925534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796d37f-0b1f-4ac8-8c62-090c8691638d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:07.927586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796d37f-0b1f-4ac8-8c62-090c8691638d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:07.954219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1157/2326] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:01:07.958454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246c142-7174-4e6c-a37d-d25009af428a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:07.997629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246c142-7174-4e6c-a37d-d25009af428a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:08.003394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722fd2b-3bf1-455e-966a-a62c03521acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:08.004536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722fd2b-3bf1-455e-966a-a62c03521acb admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:08.004991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722fd2b-3bf1-455e-966a-a62c03521acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:08.005554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722fd2b-3bf1-455e-966a-a62c03521acb admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:08.006763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722fd2b-3bf1-455e-966a-a62c03521acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:08.009651 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722fd2b-3bf1-455e-966a-a62c03521acb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:08.061272 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1170/2327] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:01:07 2022] POST /identity/v3/projects => generated 399 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:01:08.066175 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796d37f-0b1f-4ac8-8c62-090c8691638d admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:08.081041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796d37f-0b1f-4ac8-8c62-090c8691638d admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:08.282690 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57923eb9-e385-4a1c-9235-692eb497a1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:08.283542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57923eb9-e385-4a1c-9235-692eb497a1ff admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:08.283931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57923eb9-e385-4a1c-9235-692eb497a1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:08.284311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57923eb9-e385-4a1c-9235-692eb497a1ff admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:08.285694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57923eb9-e385-4a1c-9235-692eb497a1ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:08.287252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57923eb9-e385-4a1c-9235-692eb497a1ff admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:08.348217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1158/2328] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:01:08 2022] POST /identity/v3/users => generated 300 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:01:08.348994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722fd2b-3bf1-455e-966a-a62c03521acb admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:08.370179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722fd2b-3bf1-455e-966a-a62c03521acb admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:08.403217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6940b4-d187-4e9a-b395-c3fae5d6082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:08.403217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6940b4-d187-4e9a-b395-c3fae5d6082f admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:08.403217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6940b4-d187-4e9a-b395-c3fae5d6082f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:08.403217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6940b4-d187-4e9a-b395-c3fae5d6082f admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:08.403889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6940b4-d187-4e9a-b395-c3fae5d6082f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:08.436091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6940b4-d187-4e9a-b395-c3fae5d6082f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:08.505516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1171/2329] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:01:08.507384 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57923eb9-e385-4a1c-9235-692eb497a1ff admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:08.527461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57923eb9-e385-4a1c-9235-692eb497a1ff admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:08.548405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a739cdc-6e66-4aae-abb4-11967b78d554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:08.548943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a739cdc-6e66-4aae-abb4-11967b78d554 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:08.549225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a739cdc-6e66-4aae-abb4-11967b78d554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:08.549506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a739cdc-6e66-4aae-abb4-11967b78d554 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:08.550251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a739cdc-6e66-4aae-abb4-11967b78d554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:08.552191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a739cdc-6e66-4aae-abb4-11967b78d554 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:08.571512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1159/2330] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:01:08.572730 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6940b4-d187-4e9a-b395-c3fae5d6082f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:08.598487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6940b4-d187-4e9a-b395-c3fae5d6082f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:08.620426 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2696822-c762-47c2-88ce-e81acb9b8e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:08.622017 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2696822-c762-47c2-88ce-e81acb9b8e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:08.622406 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2696822-c762-47c2-88ce-e81acb9b8e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:08.622727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2696822-c762-47c2-88ce-e81acb9b8e27 admin admin] PATH_INFO: `/v3/projects/1d85340d1316415387ce33beb385afae/users/1215ebb8dc094ba0b24fdf69729843a2/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:08.669750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2696822-c762-47c2-88ce-e81acb9b8e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d85340d1316415387ce33beb385afae, user_id=1215ebb8dc094ba0b24fdf69729843a2, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:08.680233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2696822-c762-47c2-88ce-e81acb9b8e27 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:08.775816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1172/2331] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:01:08 2022] PUT /identity/v3/projects/1d85340d1316415387ce33beb385afae/users/1215ebb8dc094ba0b24fdf69729843a2/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:01:08.787269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f0a59-5a2a-4218-a1fb-ddf3b0ddc994 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:08.787735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f0a59-5a2a-4218-a1fb-ddf3b0ddc994 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:08.787830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f0a59-5a2a-4218-a1fb-ddf3b0ddc994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:08.891700 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-4b0f0a59-5a2a-4218-a1fb-ddf3b0ddc994 None None] MFA Rules not processed for user `1215ebb8dc094ba0b24fdf69729843a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:08.893268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4b0f0a59-5a2a-4218-a1fb-ddf3b0ddc994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:09.088886 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1160/2332] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:09.103924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2696822-c762-47c2-88ce-e81acb9b8e27 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:09.105890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b2696822-c762-47c2-88ce-e81acb9b8e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:09.156721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:09.215795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2696822-c762-47c2-88ce-e81acb9b8e27 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:09.225487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed800d9-bd72-484d-9cb7-5599292ef9d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:09.226275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed800d9-bd72-484d-9cb7-5599292ef9d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:09.228195 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed800d9-bd72-484d-9cb7-5599292ef9d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:09.228488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed800d9-bd72-484d-9cb7-5599292ef9d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:09.245336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed800d9-bd72-484d-9cb7-5599292ef9d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:09.249930 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed800d9-bd72-484d-9cb7-5599292ef9d7 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:09.263892 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1173/2333] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:16.984532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5af9-2368-4b7f-9e78-297328913b96 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:16.985226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5af9-2368-4b7f-9e78-297328913b96 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:16.985226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5af9-2368-4b7f-9e78-297328913b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:16.998439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:17.005501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-9d6a5af9-2368-4b7f-9e78-297328913b96 None None] MFA Rules not processed for user `f2ed196e5ad6415aa9b7a7c1b3dad277`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:17.007899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9d6a5af9-2368-4b7f-9e78-297328913b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:17.067821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1161/2334] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:19.258113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=13 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:19.425632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed800d9-bd72-484d-9cb7-5599292ef9d7 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:19.493216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed800d9-bd72-484d-9cb7-5599292ef9d7 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:19.498193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:19.499135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:19.499469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:19.499750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] PATH_INFO: `/v3/users/1215ebb8dc094ba0b24fdf69729843a2` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:19.501104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1215ebb8dc094ba0b24fdf69729843a2)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:19.503122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:19.554006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1215ebb8dc094ba0b24fdf69729843a2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:19.566130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1215ebb8dc094ba0b24fdf69729843a2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:19.566936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1215ebb8dc094ba0b24fdf69729843a2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:19.575606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1215ebb8dc094ba0b24fdf69729843a2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:19.576414 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1215ebb8dc094ba0b24fdf69729843a2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:19.592805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1174/2335] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:01:19 2022] DELETE /identity/v3/users/1215ebb8dc094ba0b24fdf69729843a2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:01:19.597720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8a5a4-f1a7-47c6-8aaa-22fb2278dea2 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:19.598398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8a5a4-f1a7-47c6-8aaa-22fb2278dea2 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:19.598724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8a5a4-f1a7-47c6-8aaa-22fb2278dea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:19.628936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-94a8a5a4-f1a7-47c6-8aaa-22fb2278dea2 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:19.630170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-94a8a5a4-f1a7-47c6-8aaa-22fb2278dea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:19.695744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1162/2336] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:01:19.705606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:19.707096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:19.770582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428b5b9-713f-4f40-b86d-c012066d2440 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:19.774262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24319b1-8541-4db2-8aaa-258fa422fb9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:19.774954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24319b1-8541-4db2-8aaa-258fa422fb9e service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:19.775237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24319b1-8541-4db2-8aaa-258fa422fb9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:19.775510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24319b1-8541-4db2-8aaa-258fa422fb9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:19.785105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24319b1-8541-4db2-8aaa-258fa422fb9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:19.786555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24319b1-8541-4db2-8aaa-258fa422fb9e service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:19.799464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1175/2337] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:20.089127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8a5a4-f1a7-47c6-8aaa-22fb2278dea2 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:20.091358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-94a8a5a4-f1a7-47c6-8aaa-22fb2278dea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:20.106411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8a5a4-f1a7-47c6-8aaa-22fb2278dea2 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:20.122072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:20.124141 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:20.124425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:20.124691 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] PATH_INFO: `/v3/projects/1d85340d1316415387ce33beb385afae` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:20.126865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d85340d1316415387ce33beb385afae)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:20.129596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:20.190296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d85340d1316415387ce33beb385afae', 'request_id': 'req-5ca9515f-7c51-482c-a68e-91a37b90555f'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:20.200426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d85340d1316415387ce33beb385afae', 'request_id': 'req-5ca9515f-7c51-482c-a68e-91a37b90555f'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:20.207204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d85340d1316415387ce33beb385afae', 'request_id': 'req-5ca9515f-7c51-482c-a68e-91a37b90555f'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:20.225415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] The token cache is being invalidate because project 1d85340d1316415387ce33beb385afae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:01:20.225845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:20.226520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:20.230600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1163/2338] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:01:20 2022] DELETE /identity/v3/projects/1d85340d1316415387ce33beb385afae => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:01:21.696907 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960cded-0ed9-483f-8752-c66d8fa75c78 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:21.697926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960cded-0ed9-483f-8752-c66d8fa75c78 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:21.698347 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960cded-0ed9-483f-8752-c66d8fa75c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:21.721831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1960cded-0ed9-483f-8752-c66d8fa75c78 None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:21.723181 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1960cded-0ed9-483f-8752-c66d8fa75c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:21.755642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:21.863803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1176/2339] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 10:01:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:21.888552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:21.890954 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:21.996350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:22.054554 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9515f-7c51-482c-a68e-91a37b90555f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:22.062774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f8614-85fe-47d2-af4b-0e51111a8472 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:22.063738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f8614-85fe-47d2-af4b-0e51111a8472 service swift] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:22.064083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f8614-85fe-47d2-af4b-0e51111a8472 service swift] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:22.064428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f8614-85fe-47d2-af4b-0e51111a8472 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:22.097527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f8614-85fe-47d2-af4b-0e51111a8472 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:22.099574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f8614-85fe-47d2-af4b-0e51111a8472 service swift] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:22.117652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1164/2340] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:01:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:01:23.095672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445f8a2-0f57-4c40-8a85-2cf9412ec549 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:23.096277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445f8a2-0f57-4c40-8a85-2cf9412ec549 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:23.096277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445f8a2-0f57-4c40-8a85-2cf9412ec549 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:23.097711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1177/2341] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 10:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 10:01:23.114031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd3cba-02fe-42ee-9113-2db011e4bba6 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:23.114643 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd3cba-02fe-42ee-9113-2db011e4bba6 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:23.114960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd3cba-02fe-42ee-9113-2db011e4bba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:23.234635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-d8dd3cba-02fe-42ee-9113-2db011e4bba6 None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:23.249771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d8dd3cba-02fe-42ee-9113-2db011e4bba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:23.431652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1165/2342] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 10:01:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:01:23.448617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445f8a2-0f57-4c40-8a85-2cf9412ec549 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:23.451023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d445f8a2-0f57-4c40-8a85-2cf9412ec549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:23.540411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445f8a2-0f57-4c40-8a85-2cf9412ec549 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:23.548328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c240e9-a050-4e02-840a-b6d458c41b4c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e850be24df24bcf972e6e31002014cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:23.549921 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c240e9-a050-4e02-840a-b6d458c41b4c service glance] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:23.550412 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c240e9-a050-4e02-840a-b6d458c41b4c service glance] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:23.550972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c240e9-a050-4e02-840a-b6d458c41b4c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:23.565642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c240e9-a050-4e02-840a-b6d458c41b4c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:23.567527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c240e9-a050-4e02-840a-b6d458c41b4c service glance] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:23.582922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1178/2343] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:23.662696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9b2b5-9bf5-4259-843a-e4c801bd4907 None None] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:23.663179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9b2b5-9bf5-4259-843a-e4c801bd4907 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:23.663543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9b2b5-9bf5-4259-843a-e4c801bd4907 None None] PATH_INFO: `/` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:23.665641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1166/2344] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 10:01:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 10:01:23.673335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69102791-2286-4c92-aef2-a18029183f8a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:23.673625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69102791-2286-4c92-aef2-a18029183f8a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:23.673901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69102791-2286-4c92-aef2-a18029183f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:23.726704 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-69102791-2286-4c92-aef2-a18029183f8a None None] MFA Rules not processed for user `4c97b321bf854177bd2f310cf2958959`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:23.727943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-69102791-2286-4c92-aef2-a18029183f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:23.793425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1179/2345] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 10:01:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:23.805910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9b2b5-9bf5-4259-843a-e4c801bd4907 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:23.807688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-83e9b2b5-9bf5-4259-843a-e4c801bd4907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:23.821740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9b2b5-9bf5-4259-843a-e4c801bd4907 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:23.825654 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c42e7-d41e-41be-a141-02106c7febb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:23.826368 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c42e7-d41e-41be-a141-02106c7febb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:23.826709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c42e7-d41e-41be-a141-02106c7febb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:23.827015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c42e7-d41e-41be-a141-02106c7febb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:23.836136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c42e7-d41e-41be-a141-02106c7febb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:23.837618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c42e7-d41e-41be-a141-02106c7febb9 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:23.860640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1167/2346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:23.941575 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca98b66-65e2-4ee5-ad10-189cce048fbf None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:23.941892 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca98b66-65e2-4ee5-ad10-189cce048fbf None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:23.942188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca98b66-65e2-4ee5-ad10-189cce048fbf None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:23.943666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1180/2347] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 10:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 10:01:23.949780 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b23fb-9714-400c-b1ab-21dd1eea6342 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:23.950264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b23fb-9714-400c-b1ab-21dd1eea6342 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:23.951009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b23fb-9714-400c-b1ab-21dd1eea6342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:23.973425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2a4b23fb-9714-400c-b1ab-21dd1eea6342 None None] MFA Rules not processed for user `8a5eea7f539048c48ed10e72dd933df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:23.974748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2a4b23fb-9714-400c-b1ab-21dd1eea6342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:23.985635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1168/2348] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 10:01:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:23.997036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca98b66-65e2-4ee5-ad10-189cce048fbf None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:24.025603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca98b66-65e2-4ee5-ad10-189cce048fbf None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:24.031623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb140e-b529-4e41-a343-b51765e24842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:24.032460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb140e-b529-4e41-a343-b51765e24842 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:24.032807 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb140e-b529-4e41-a343-b51765e24842 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:24.033149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb140e-b529-4e41-a343-b51765e24842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:24.048125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb140e-b529-4e41-a343-b51765e24842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:24.052358 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb140e-b529-4e41-a343-b51765e24842 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:24.072782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1181/2349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:33.967730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=71 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:34.032774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b23fb-9714-400c-b1ab-21dd1eea6342 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:34.034821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2a4b23fb-9714-400c-b1ab-21dd1eea6342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:34.066219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=73 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:34.150140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b23fb-9714-400c-b1ab-21dd1eea6342 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:34.155873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b842b4-dfce-43d2-a74e-c08cc1284187 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:34.156605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b842b4-dfce-43d2-a74e-c08cc1284187 service placement] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:34.156864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b842b4-dfce-43d2-a74e-c08cc1284187 service placement] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:34.157217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b842b4-dfce-43d2-a74e-c08cc1284187 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:34.160347 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f2b842b4-dfce-43d2-a74e-c08cc1284187 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:34.177469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b842b4-dfce-43d2-a74e-c08cc1284187 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:34.179088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b842b4-dfce-43d2-a74e-c08cc1284187 service placement] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:34.190911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1169/2350] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:37.725540 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73186217-fd87-4274-be99-566851f701ed None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:37.726119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73186217-fd87-4274-be99-566851f701ed None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:37.726119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73186217-fd87-4274-be99-566851f701ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:37.783584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-73186217-fd87-4274-be99-566851f701ed None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:37.784829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-73186217-fd87-4274-be99-566851f701ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:37.806532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1182/2351] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 10:01:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:37.823482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b842b4-dfce-43d2-a74e-c08cc1284187 service placement] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:37.846279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b842b4-dfce-43d2-a74e-c08cc1284187 service placement] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:37.853235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35abf27-0188-4f30-bb6f-efed530fb9c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:37.853899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35abf27-0188-4f30-bb6f-efed530fb9c3 service swift] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:37.854176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35abf27-0188-4f30-bb6f-efed530fb9c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:37.854443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35abf27-0188-4f30-bb6f-efed530fb9c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:37.863555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35abf27-0188-4f30-bb6f-efed530fb9c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:37.865135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35abf27-0188-4f30-bb6f-efed530fb9c3 service swift] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:37.874501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1170/2352] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:01:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:01:46.422145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73186217-fd87-4274-be99-566851f701ed None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:46.424118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-73186217-fd87-4274-be99-566851f701ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:46.531571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73186217-fd87-4274-be99-566851f701ed None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:46.536961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:46.538240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:46.538719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:46.539192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] PATH_INFO: `/v3/users/6dbb8c0ccb274fbfb460d0afda35802c` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:46.540684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dbb8c0ccb274fbfb460d0afda35802c)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:46.542786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:46.601045 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dbb8c0ccb274fbfb460d0afda35802c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:46.613818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dbb8c0ccb274fbfb460d0afda35802c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:46.614557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dbb8c0ccb274fbfb460d0afda35802c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:46.641355 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dbb8c0ccb274fbfb460d0afda35802c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:46.651566 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dbb8c0ccb274fbfb460d0afda35802c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:46.675080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1183/2353] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:01:46 2022] DELETE /identity/v3/users/6dbb8c0ccb274fbfb460d0afda35802c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:01:46.681927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61090a9-b80b-4fd1-a935-c7b5d459c506 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:46.682605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61090a9-b80b-4fd1-a935-c7b5d459c506 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:46.683393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61090a9-b80b-4fd1-a935-c7b5d459c506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:46.712078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b61090a9-b80b-4fd1-a935-c7b5d459c506 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:46.714297 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b61090a9-b80b-4fd1-a935-c7b5d459c506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:46.815861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1171/2354] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:46.826258 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:46.827944 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:46.909951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:46.943316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca75ea-7115-4804-a662-4593ca4a5139 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:46.949188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb107586-66e6-4657-bda9-5203d84f7bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:46.950124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb107586-66e6-4657-bda9-5203d84f7bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:46.951400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb107586-66e6-4657-bda9-5203d84f7bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:46.951400 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb107586-66e6-4657-bda9-5203d84f7bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:46.977503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb107586-66e6-4657-bda9-5203d84f7bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:46.977503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb107586-66e6-4657-bda9-5203d84f7bb3 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:46.988592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1184/2355] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:47.376387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61090a9-b80b-4fd1-a935-c7b5d459c506 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:47.377932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b61090a9-b80b-4fd1-a935-c7b5d459c506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:47.390204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61090a9-b80b-4fd1-a935-c7b5d459c506 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:47.400622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:47.401878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:47.402157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:47.402428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] PATH_INFO: `/v3/projects/442f757fbece4bf8a0021dd4047fa468` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:47.410987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=442f757fbece4bf8a0021dd4047fa468)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:47.413219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:47.457764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:47.462200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442f757fbece4bf8a0021dd4047fa468', 'request_id': 'req-ca0828fa-e122-4407-a177-666ccfeb66b3'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:47.470178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442f757fbece4bf8a0021dd4047fa468', 'request_id': 'req-ca0828fa-e122-4407-a177-666ccfeb66b3'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:47.476408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442f757fbece4bf8a0021dd4047fa468', 'request_id': 'req-ca0828fa-e122-4407-a177-666ccfeb66b3'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:47.487615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] The token cache is being invalidate because project 442f757fbece4bf8a0021dd4047fa468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:01:47.487921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:47.488618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ca0828fa-e122-4407-a177-666ccfeb66b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:01:47.490215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1172/2356] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:01:47 2022] DELETE /identity/v3/projects/442f757fbece4bf8a0021dd4047fa468 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:01:47.495902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3068cb3-e884-46b4-b49f-2005d4b86d69 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:47.496449 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3068cb3-e884-46b4-b49f-2005d4b86d69 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:47.496449 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3068cb3-e884-46b4-b49f-2005d4b86d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:47.514884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a3068cb3-e884-46b4-b49f-2005d4b86d69 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:47.516077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a3068cb3-e884-46b4-b49f-2005d4b86d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:47.579523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1185/2357] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:47.588268 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea7c2b-63fe-4f5b-bc07-e883cd5b7883 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:47.588552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea7c2b-63fe-4f5b-bc07-e883cd5b7883 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:47.588952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea7c2b-63fe-4f5b-bc07-e883cd5b7883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:47.607914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-e8ea7c2b-63fe-4f5b-bc07-e883cd5b7883 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:47.609140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e8ea7c2b-63fe-4f5b-bc07-e883cd5b7883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:47.617212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1173/2358] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:47.621605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3068cb3-e884-46b4-b49f-2005d4b86d69 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:47.631844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3068cb3-e884-46b4-b49f-2005d4b86d69 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:47.638032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024b5f7-52dd-4816-aed8-c7100e400336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:47.639408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024b5f7-52dd-4816-aed8-c7100e400336 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:47.639885 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024b5f7-52dd-4816-aed8-c7100e400336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:47.640225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024b5f7-52dd-4816-aed8-c7100e400336 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:47.641432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024b5f7-52dd-4816-aed8-c7100e400336 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:01:47.641853 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024b5f7-52dd-4816-aed8-c7100e400336 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:47.644457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024b5f7-52dd-4816-aed8-c7100e400336 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:47.658604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1186/2359] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:01:47.663537 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea7c2b-63fe-4f5b-bc07-e883cd5b7883 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:47.673518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea7c2b-63fe-4f5b-bc07-e883cd5b7883 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:47.678785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b40e70-eda5-46d1-b499-647ece5b9e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:47.679445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b40e70-eda5-46d1-b499-647ece5b9e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:47.679786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b40e70-eda5-46d1-b499-647ece5b9e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:47.679991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b40e70-eda5-46d1-b499-647ece5b9e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:47.680683 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b40e70-eda5-46d1-b499-647ece5b9e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:47.682608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b40e70-eda5-46d1-b499-647ece5b9e3e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:47.711626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1174/2360] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:01:47 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:01:47.715937 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024b5f7-52dd-4816-aed8-c7100e400336 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:47.727166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024b5f7-52dd-4816-aed8-c7100e400336 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:47.797386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ae63e-9619-40ba-8b28-30763d6ba9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:47.798083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ae63e-9619-40ba-8b28-30763d6ba9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:47.798365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ae63e-9619-40ba-8b28-30763d6ba9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:47.798672 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ae63e-9619-40ba-8b28-30763d6ba9f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:47.799403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ae63e-9619-40ba-8b28-30763d6ba9f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:47.801361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ae63e-9619-40ba-8b28-30763d6ba9f4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:47.823516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1187/2361] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:01:47 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:01:47.827679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b40e70-eda5-46d1-b499-647ece5b9e3e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:47.839928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b40e70-eda5-46d1-b499-647ece5b9e3e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:47.845548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbabbd4-817d-45e4-ac0d-5506b34d8253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:47.846320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbabbd4-817d-45e4-ac0d-5506b34d8253 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:47.846654 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbabbd4-817d-45e4-ac0d-5506b34d8253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:47.847013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbabbd4-817d-45e4-ac0d-5506b34d8253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:47.847749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbabbd4-817d-45e4-ac0d-5506b34d8253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:47.849659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbabbd4-817d-45e4-ac0d-5506b34d8253 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:47.861472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1175/2362] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:01:47.865604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ae63e-9619-40ba-8b28-30763d6ba9f4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:47.876491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ae63e-9619-40ba-8b28-30763d6ba9f4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:47.881708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f20e8-4dd6-41b0-a52e-b442444e7450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:47.882393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f20e8-4dd6-41b0-a52e-b442444e7450 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:47.882712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f20e8-4dd6-41b0-a52e-b442444e7450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:47.882992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f20e8-4dd6-41b0-a52e-b442444e7450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:47.883730 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f20e8-4dd6-41b0-a52e-b442444e7450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:47.885378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f20e8-4dd6-41b0-a52e-b442444e7450 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:47.898665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1188/2363] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:01:47.902968 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbabbd4-817d-45e4-ac0d-5506b34d8253 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:47.913829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbabbd4-817d-45e4-ac0d-5506b34d8253 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:47.919332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418d6bb-a5d6-4114-a2f1-b02bb7b236db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:47.920495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418d6bb-a5d6-4114-a2f1-b02bb7b236db admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:47.920772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418d6bb-a5d6-4114-a2f1-b02bb7b236db admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:47.921066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418d6bb-a5d6-4114-a2f1-b02bb7b236db admin admin] PATH_INFO: `/v3/projects/4779579fbd6f42dcb6ad98e752ef741c/users/43964a0e6b48487bb90c5356d312b60d/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:47.937702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418d6bb-a5d6-4114-a2f1-b02bb7b236db admin admin] RBAC: Authorizing `identity:create_grant(project_id=4779579fbd6f42dcb6ad98e752ef741c, user_id=43964a0e6b48487bb90c5356d312b60d, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:47.940463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418d6bb-a5d6-4114-a2f1-b02bb7b236db admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:47.955235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1176/2364] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:01:47 2022] PUT /identity/v3/projects/4779579fbd6f42dcb6ad98e752ef741c/users/43964a0e6b48487bb90c5356d312b60d/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:01:47.960714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50e1d9-b57c-4e04-9e85-cbfea429c8f1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:47.961160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50e1d9-b57c-4e04-9e85-cbfea429c8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:47.961441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50e1d9-b57c-4e04-9e85-cbfea429c8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:47.989014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ff50e1d9-b57c-4e04-9e85-cbfea429c8f1 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:47.990262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ff50e1d9-b57c-4e04-9e85-cbfea429c8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:48.048124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:48.074538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1189/2365] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:48.086189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418d6bb-a5d6-4114-a2f1-b02bb7b236db admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:48.087926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7418d6bb-a5d6-4114-a2f1-b02bb7b236db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:48.157276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418d6bb-a5d6-4114-a2f1-b02bb7b236db admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:48.161206 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceadc2-20ac-4bfe-9042-3fc724e8e306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:48.161872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceadc2-20ac-4bfe-9042-3fc724e8e306 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:48.162149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceadc2-20ac-4bfe-9042-3fc724e8e306 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:48.162422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceadc2-20ac-4bfe-9042-3fc724e8e306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:48.173282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ceadc2-20ac-4bfe-9042-3fc724e8e306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:48.174728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ceadc2-20ac-4bfe-9042-3fc724e8e306 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:48.185977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1177/2366] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:48.918886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969bd8-1cef-4759-9410-8282b1798413 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:48.919436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969bd8-1cef-4759-9410-8282b1798413 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:48.919436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969bd8-1cef-4759-9410-8282b1798413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:48.950356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f2969bd8-1cef-4759-9410-8282b1798413 None None] MFA Rules not processed for user `43964a0e6b48487bb90c5356d312b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:48.951656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f2969bd8-1cef-4759-9410-8282b1798413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:49.066153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1190/2367] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:49.078771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceadc2-20ac-4bfe-9042-3fc724e8e306 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:01:49.080754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-69ceadc2-20ac-4bfe-9042-3fc724e8e306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:49.178550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceadc2-20ac-4bfe-9042-3fc724e8e306 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:01:49.183385 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a2a60-dca6-4b4d-af96-7a3c4c6dbd4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:01:49.184128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a2a60-dca6-4b4d-af96-7a3c4c6dbd4e service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:49.184478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a2a60-dca6-4b4d-af96-7a3c4c6dbd4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:49.184815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a2a60-dca6-4b4d-af96-7a3c4c6dbd4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:49.192817 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:01:49.199644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a2a60-dca6-4b4d-af96-7a3c4c6dbd4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:01:49.201658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a2a60-dca6-4b4d-af96-7a3c4c6dbd4e service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:01:49.213921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1178/2368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:01:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:01:50.665196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde18948-5b61-4ed5-9cfe-9b7a2673f9d2 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:01:50.666080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde18948-5b61-4ed5-9cfe-9b7a2673f9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:01:50.666713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde18948-5b61-4ed5-9cfe-9b7a2673f9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:01:50.701448 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-bde18948-5b61-4ed5-9cfe-9b7a2673f9d2 None None] MFA Rules not processed for user `43964a0e6b48487bb90c5356d312b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:01:50.703544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bde18948-5b61-4ed5-9cfe-9b7a2673f9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:01:50.720828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1191/2369] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:01:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:01:59.208646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:02:00.679163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=40 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:02:05.549375 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a2a60-dca6-4b4d-af96-7a3c4c6dbd4e service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:02:05.560157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9d6a2a60-dca6-4b4d-af96-7a3c4c6dbd4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:02:05.565335 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a2a60-dca6-4b4d-af96-7a3c4c6dbd4e service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:02:05.569396 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8108d1-12d7-473a-b1a0-f511491faed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:02:05.569762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8108d1-12d7-473a-b1a0-f511491faed7 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:02:05.570041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8108d1-12d7-473a-b1a0-f511491faed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:02:05.570639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8108d1-12d7-473a-b1a0-f511491faed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:02:05.572540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cf8108d1-12d7-473a-b1a0-f511491faed7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:02:05.652199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8108d1-12d7-473a-b1a0-f511491faed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:02:05.653715 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8108d1-12d7-473a-b1a0-f511491faed7 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:02:05.667833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1179/2370] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:02:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:02:06.775357 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de163fb-070d-448a-9b31-01b91816beef None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:02:06.775900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de163fb-070d-448a-9b31-01b91816beef None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:02:06.775900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de163fb-070d-448a-9b31-01b91816beef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:02:06.812021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4de163fb-070d-448a-9b31-01b91816beef None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:02:06.814863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4de163fb-070d-448a-9b31-01b91816beef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:02:06.925084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1192/2371] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jun 29 10:02:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:02:06.939707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8108d1-12d7-473a-b1a0-f511491faed7 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:02:06.941800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cf8108d1-12d7-473a-b1a0-f511491faed7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:02:07.074811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8108d1-12d7-473a-b1a0-f511491faed7 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:02:07.080906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6c57e-5d85-4e66-ba6b-5f2ab0c62633 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:02:07.082018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6c57e-5d85-4e66-ba6b-5f2ab0c62633 service swift] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:02:07.082453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6c57e-5d85-4e66-ba6b-5f2ab0c62633 service swift] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:02:07.082930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6c57e-5d85-4e66-ba6b-5f2ab0c62633 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:02:07.102079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6c57e-5d85-4e66-ba6b-5f2ab0c62633 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:02:07.103871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6c57e-5d85-4e66-ba6b-5f2ab0c62633 service swift] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:02:07.128436 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1180/2372] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:02:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:02:16.918004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=25 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:02:17.110168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=67 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:02:17.252499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb09bce-5511-4fd7-969b-1899e37f86cd None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:02:17.252753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb09bce-5511-4fd7-969b-1899e37f86cd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:02:17.253033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb09bce-5511-4fd7-969b-1899e37f86cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:02:17.272962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2fb09bce-5511-4fd7-969b-1899e37f86cd None None] MFA Rules not processed for user `229771b215b3492abc2c5ec38ff0d929`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:02:17.274178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2fb09bce-5511-4fd7-969b-1899e37f86cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:02:17.335780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1193/2373] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:02:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:02:27.328201 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=25 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:03:19.810270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6c57e-5d85-4e66-ba6b-5f2ab0c62633 service swift] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:03:19.821866 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6c57e-5d85-4e66-ba6b-5f2ab0c62633 service swift] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:03:19.826937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c27881-56f6-4f73-a793-c6b10e8b8e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:03:19.827733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c27881-56f6-4f73-a793-c6b10e8b8e6d service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:03:19.828066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c27881-56f6-4f73-a793-c6b10e8b8e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:03:19.828391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c27881-56f6-4f73-a793-c6b10e8b8e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:03:19.830694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a1c27881-56f6-4f73-a793-c6b10e8b8e6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:03:19.917470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c27881-56f6-4f73-a793-c6b10e8b8e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:03:19.918994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c27881-56f6-4f73-a793-c6b10e8b8e6d service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:03:19.931889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1181/2374] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:03:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:03:21.333054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb09bce-5511-4fd7-969b-1899e37f86cd None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:03:21.345063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb09bce-5511-4fd7-969b-1899e37f86cd None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:03:21.350859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568a2d5-a247-4dd8-9d28-1662ff647232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:03:21.351785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568a2d5-a247-4dd8-9d28-1662ff647232 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:03:21.352297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568a2d5-a247-4dd8-9d28-1662ff647232 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:03:21.352605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568a2d5-a247-4dd8-9d28-1662ff647232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:03:21.355424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f568a2d5-a247-4dd8-9d28-1662ff647232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:03:21.374263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568a2d5-a247-4dd8-9d28-1662ff647232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:03:21.376688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568a2d5-a247-4dd8-9d28-1662ff647232 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:03:21.392341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1194/2375] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:03:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:03:28.913470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c27881-56f6-4f73-a793-c6b10e8b8e6d service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:03:28.915411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a1c27881-56f6-4f73-a793-c6b10e8b8e6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:03:28.930870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c27881-56f6-4f73-a793-c6b10e8b8e6d service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:03:28.934690 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53991211-6bdc-4f44-8298-39dcdf540336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:03:28.935395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53991211-6bdc-4f44-8298-39dcdf540336 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:03:28.935675 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53991211-6bdc-4f44-8298-39dcdf540336 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:03:28.935950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53991211-6bdc-4f44-8298-39dcdf540336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:03:28.937853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-53991211-6bdc-4f44-8298-39dcdf540336 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:03:28.951171 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53991211-6bdc-4f44-8298-39dcdf540336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:03:28.952652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53991211-6bdc-4f44-8298-39dcdf540336 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:03:28.966861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1182/2376] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:03:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:03:31.386556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:03:33.891637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568a2d5-a247-4dd8-9d28-1662ff647232 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:03:33.893217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f568a2d5-a247-4dd8-9d28-1662ff647232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:03:33.909362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568a2d5-a247-4dd8-9d28-1662ff647232 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:03:33.916924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46517f76-4043-422b-aafd-09f4d1b29c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:03:33.917607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46517f76-4043-422b-aafd-09f4d1b29c0b service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:03:33.917889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46517f76-4043-422b-aafd-09f4d1b29c0b service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:03:33.918161 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46517f76-4043-422b-aafd-09f4d1b29c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:03:33.920552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-46517f76-4043-422b-aafd-09f4d1b29c0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:03:33.999889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46517f76-4043-422b-aafd-09f4d1b29c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:03:34.003048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46517f76-4043-422b-aafd-09f4d1b29c0b service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:03:34.020013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1195/2377] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:03:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:03:38.958399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=40 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:03:40.379272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53991211-6bdc-4f44-8298-39dcdf540336 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:03:40.393903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53991211-6bdc-4f44-8298-39dcdf540336 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:03:40.401416 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc497fb4-4789-4542-bef4-2c7a30319b98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:03:40.402165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc497fb4-4789-4542-bef4-2c7a30319b98 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:03:40.402748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc497fb4-4789-4542-bef4-2c7a30319b98 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:03:40.402925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc497fb4-4789-4542-bef4-2c7a30319b98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:03:40.405121 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cc497fb4-4789-4542-bef4-2c7a30319b98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:03:40.484182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc497fb4-4789-4542-bef4-2c7a30319b98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:03:40.485854 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc497fb4-4789-4542-bef4-2c7a30319b98 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:03:40.497703 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1183/2378] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:03:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:03:44.012769 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=28 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:03:50.492188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=28 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:03:56.599046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bc742-d4c5-40b8-a725-69396678fd0e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:03:56.599692 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bc742-d4c5-40b8-a725-69396678fd0e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:03:56.599800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bc742-d4c5-40b8-a725-69396678fd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:03:56.622013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fa6bc742-d4c5-40b8-a725-69396678fd0e None None] MFA Rules not processed for user `63ba0760d13e4bbda0650de803ef46ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:03:56.623906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fa6bc742-d4c5-40b8-a725-69396678fd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:03:56.633841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1196/2379] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:03:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:04:06.615465 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:04:09.343718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc497fb4-4789-4542-bef4-2c7a30319b98 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:04:09.345941 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cc497fb4-4789-4542-bef4-2c7a30319b98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:04:09.366399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc497fb4-4789-4542-bef4-2c7a30319b98 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:04:09.373536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec12f40-238f-48ac-a894-2e65158b164e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:04:09.374418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec12f40-238f-48ac-a894-2e65158b164e service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:04:09.374914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec12f40-238f-48ac-a894-2e65158b164e service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:04:09.375339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec12f40-238f-48ac-a894-2e65158b164e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:04:09.378033 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-eec12f40-238f-48ac-a894-2e65158b164e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:04:09.393032 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec12f40-238f-48ac-a894-2e65158b164e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:04:09.394619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec12f40-238f-48ac-a894-2e65158b164e service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:04:09.411207 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1184/2380] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:04:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:04:09.484896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bc742-d4c5-40b8-a725-69396678fd0e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:04:09.495417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bc742-d4c5-40b8-a725-69396678fd0e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:04:09.501192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e5ae6-8e23-4c4a-9afa-b58f9632ffe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:04:09.501867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e5ae6-8e23-4c4a-9afa-b58f9632ffe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:04:09.502360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e5ae6-8e23-4c4a-9afa-b58f9632ffe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:04:09.502844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e5ae6-8e23-4c4a-9afa-b58f9632ffe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:04:09.504920 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-001e5ae6-8e23-4c4a-9afa-b58f9632ffe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:04:09.515727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e5ae6-8e23-4c4a-9afa-b58f9632ffe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:04:09.517436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e5ae6-8e23-4c4a-9afa-b58f9632ffe1 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:04:09.528127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1197/2381] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:04:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:04:19.405776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=6 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:04:19.523636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:05:36.619259 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec12f40-238f-48ac-a894-2e65158b164e service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:05:36.631543 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec12f40-238f-48ac-a894-2e65158b164e service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:05:36.638411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b16e4-7ec3-4c00-a5ce-5240283dfcf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:05:36.639078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b16e4-7ec3-4c00-a5ce-5240283dfcf3 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:05:36.639359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b16e4-7ec3-4c00-a5ce-5240283dfcf3 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:05:36.639627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b16e4-7ec3-4c00-a5ce-5240283dfcf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:05:36.641566 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-382b16e4-7ec3-4c00-a5ce-5240283dfcf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:05:36.710641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b16e4-7ec3-4c00-a5ce-5240283dfcf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:05:36.712815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b16e4-7ec3-4c00-a5ce-5240283dfcf3 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:05:36.722497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1185/2382] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:05:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:05:43.276545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e5ae6-8e23-4c4a-9afa-b58f9632ffe1 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:05:43.291020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e5ae6-8e23-4c4a-9afa-b58f9632ffe1 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:05:43.297413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17d286-177e-44da-a743-a8e4c9542a53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:05:43.298332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17d286-177e-44da-a743-a8e4c9542a53 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:05:43.299590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17d286-177e-44da-a743-a8e4c9542a53 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:05:43.299870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17d286-177e-44da-a743-a8e4c9542a53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:05:43.302336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6e17d286-177e-44da-a743-a8e4c9542a53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:05:43.315512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17d286-177e-44da-a743-a8e4c9542a53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:05:43.317131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17d286-177e-44da-a743-a8e4c9542a53 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:05:43.328296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1198/2383] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:05:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:05:46.718014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=28 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:05:53.323974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:06:10.543890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b16e4-7ec3-4c00-a5ce-5240283dfcf3 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:06:10.557559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b16e4-7ec3-4c00-a5ce-5240283dfcf3 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:06:10.564450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d53332-f241-4ee7-ad5e-bf9432fe6073 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:06:10.565091 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d53332-f241-4ee7-ad5e-bf9432fe6073 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:06:10.565367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d53332-f241-4ee7-ad5e-bf9432fe6073 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:06:10.565628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d53332-f241-4ee7-ad5e-bf9432fe6073 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:06:10.567823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-05d53332-f241-4ee7-ad5e-bf9432fe6073 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:06:10.630758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d53332-f241-4ee7-ad5e-bf9432fe6073 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:06:10.632756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d53332-f241-4ee7-ad5e-bf9432fe6073 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:06:10.642658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1186/2384] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:06:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:06:20.637811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=28 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:07:33.148250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17d286-177e-44da-a743-a8e4c9542a53 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:07:33.150245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6e17d286-177e-44da-a743-a8e4c9542a53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:07:33.232230 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17d286-177e-44da-a743-a8e4c9542a53 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:07:33.237535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1849dd7-ae4c-44cf-bd2f-bdb0fcadd2af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:07:33.238214 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1849dd7-ae4c-44cf-bd2f-bdb0fcadd2af service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:07:33.238499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1849dd7-ae4c-44cf-bd2f-bdb0fcadd2af service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:07:33.238804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1849dd7-ae4c-44cf-bd2f-bdb0fcadd2af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:07:33.240680 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e1849dd7-ae4c-44cf-bd2f-bdb0fcadd2af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:07:33.253903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1849dd7-ae4c-44cf-bd2f-bdb0fcadd2af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:07:33.255472 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1849dd7-ae4c-44cf-bd2f-bdb0fcadd2af service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:07:33.266128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1199/2385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:07:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:07:36.804863 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d53332-f241-4ee7-ad5e-bf9432fe6073 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:07:36.817312 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d53332-f241-4ee7-ad5e-bf9432fe6073 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:07:36.822075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0948a3-e98b-42ac-b868-4e4f8cd0a138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:07:36.823062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0948a3-e98b-42ac-b868-4e4f8cd0a138 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:07:36.823305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0948a3-e98b-42ac-b868-4e4f8cd0a138 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:07:36.823913 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0948a3-e98b-42ac-b868-4e4f8cd0a138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:07:36.833682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0948a3-e98b-42ac-b868-4e4f8cd0a138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:07:36.835146 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0948a3-e98b-42ac-b868-4e4f8cd0a138 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:07:36.845328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1187/2386] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:07:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:07:43.261786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=32 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:07:46.840770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=6 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:09:33.862167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1849dd7-ae4c-44cf-bd2f-bdb0fcadd2af service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:09:33.876842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1849dd7-ae4c-44cf-bd2f-bdb0fcadd2af service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:09:33.882970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82c0cf-160e-4477-abb1-c97039b9c0c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:09:33.883662 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82c0cf-160e-4477-abb1-c97039b9c0c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:09:33.883942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82c0cf-160e-4477-abb1-c97039b9c0c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:09:33.884194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82c0cf-160e-4477-abb1-c97039b9c0c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:09:33.892641 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82c0cf-160e-4477-abb1-c97039b9c0c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:09:33.894378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82c0cf-160e-4477-abb1-c97039b9c0c9 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:09:33.904887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1200/2387] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:09:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:09:37.128588 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0948a3-e98b-42ac-b868-4e4f8cd0a138 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:09:37.138657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0948a3-e98b-42ac-b868-4e4f8cd0a138 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:09:37.147829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c3a83-d8bd-42b5-8d63-7729fcb5edbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:09:37.148520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c3a83-d8bd-42b5-8d63-7729fcb5edbd service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:09:37.148820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c3a83-d8bd-42b5-8d63-7729fcb5edbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:09:37.149088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c3a83-d8bd-42b5-8d63-7729fcb5edbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:09:37.161624 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c3a83-d8bd-42b5-8d63-7729fcb5edbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:09:37.163175 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c3a83-d8bd-42b5-8d63-7729fcb5edbd service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:09:37.175846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1188/2388] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:09:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:09:37.227184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82c0cf-160e-4477-abb1-c97039b9c0c9 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:09:37.236802 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82c0cf-160e-4477-abb1-c97039b9c0c9 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:09:37.240637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6ab5a-6b32-4325-938b-88ceec13eb4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:09:37.241277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6ab5a-6b32-4325-938b-88ceec13eb4a service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:09:37.241660 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6ab5a-6b32-4325-938b-88ceec13eb4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:09:37.241959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6ab5a-6b32-4325-938b-88ceec13eb4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:09:37.253050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6ab5a-6b32-4325-938b-88ceec13eb4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:09:37.254494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6ab5a-6b32-4325-938b-88ceec13eb4a service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:09:37.265753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1201/2389] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:09:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:09:37.273476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c3a83-d8bd-42b5-8d63-7729fcb5edbd service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:09:37.282765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c3a83-d8bd-42b5-8d63-7729fcb5edbd service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:09:37.286909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705c28e-7687-466a-aea6-bb709935227b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:09:37.287566 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705c28e-7687-466a-aea6-bb709935227b service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:09:37.287840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705c28e-7687-466a-aea6-bb709935227b service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:09:37.288129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705c28e-7687-466a-aea6-bb709935227b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:09:37.298407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705c28e-7687-466a-aea6-bb709935227b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:09:37.299879 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705c28e-7687-466a-aea6-bb709935227b service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:09:37.332307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1189/2390] 10.222.0.52 () {60 vars in 1396 bytes} [Wed Jun 29 10:09:37 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:09:37.402898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6ab5a-6b32-4325-938b-88ceec13eb4a service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:09:37.412557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6ab5a-6b32-4325-938b-88ceec13eb4a service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:09:37.416660 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf8c05-4e78-40fd-afa2-9f4d50bc2837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:09:37.417349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8c05-4e78-40fd-afa2-9f4d50bc2837 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:09:37.417615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8c05-4e78-40fd-afa2-9f4d50bc2837 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:09:37.417890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8c05-4e78-40fd-afa2-9f4d50bc2837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:09:37.426595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf8c05-4e78-40fd-afa2-9f4d50bc2837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:09:37.428064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf8c05-4e78-40fd-afa2-9f4d50bc2837 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:09:37.458018 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1202/2391] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:09:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:09:37.561070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705c28e-7687-466a-aea6-bb709935227b service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:09:37.571267 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705c28e-7687-466a-aea6-bb709935227b service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:09:37.575500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e6080-1024-4ebd-8e8d-e7364d3526e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:09:37.576148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e6080-1024-4ebd-8e8d-e7364d3526e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:09:37.576424 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e6080-1024-4ebd-8e8d-e7364d3526e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:09:37.576673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e6080-1024-4ebd-8e8d-e7364d3526e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:09:37.586797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e6080-1024-4ebd-8e8d-e7364d3526e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:09:37.588155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e6080-1024-4ebd-8e8d-e7364d3526e1 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:09:37.620151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1190/2392] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:09:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:09:47.454855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=25 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:09:47.616314 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=32 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:11:37.665595 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf8c05-4e78-40fd-afa2-9f4d50bc2837 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:11:37.689957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf8c05-4e78-40fd-afa2-9f4d50bc2837 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:11:37.694082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe5a88-5cbf-47b6-9a58-7f032a1ab567 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:11:37.694768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe5a88-5cbf-47b6-9a58-7f032a1ab567 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:11:37.695170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe5a88-5cbf-47b6-9a58-7f032a1ab567 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:11:37.695498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe5a88-5cbf-47b6-9a58-7f032a1ab567 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:11:37.723326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe5a88-5cbf-47b6-9a58-7f032a1ab567 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:11:37.724794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe5a88-5cbf-47b6-9a58-7f032a1ab567 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:11:37.756459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1203/2393] 10.222.0.52 () {60 vars in 1396 bytes} [Wed Jun 29 10:11:37 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:11:38.537811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e6080-1024-4ebd-8e8d-e7364d3526e1 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:11:38.570382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe5a88-5cbf-47b6-9a58-7f032a1ab567 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:11:38.577869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe5a88-5cbf-47b6-9a58-7f032a1ab567 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:11:38.583660 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40d7de-7def-4741-b8b2-d17099de64ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:11:38.584877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d7de-7def-4741-b8b2-d17099de64ff service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:11:38.585417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d7de-7def-4741-b8b2-d17099de64ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:11:38.585695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d7de-7def-4741-b8b2-d17099de64ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:11:38.597319 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40d7de-7def-4741-b8b2-d17099de64ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:11:38.598804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40d7de-7def-4741-b8b2-d17099de64ff service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:11:38.620161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e6080-1024-4ebd-8e8d-e7364d3526e1 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:11:38.626550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fdf9c-4461-44fb-bac8-75060764ce25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:11:38.627271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fdf9c-4461-44fb-bac8-75060764ce25 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:11:38.627555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fdf9c-4461-44fb-bac8-75060764ce25 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:11:38.627831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fdf9c-4461-44fb-bac8-75060764ce25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:11:38.634741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1204/2394] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:11:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:11:38.636933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fdf9c-4461-44fb-bac8-75060764ce25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:11:38.638605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fdf9c-4461-44fb-bac8-75060764ce25 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:11:38.649723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1191/2395] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:11:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:11:48.629680 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=37 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:11:48.645646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=30 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:12:38.105080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40d7de-7def-4741-b8b2-d17099de64ff service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:12:38.106659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-cf40d7de-7def-4741-b8b2-d17099de64ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:12:38.160059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40d7de-7def-4741-b8b2-d17099de64ff service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:12:38.165816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734e6f2-2026-4974-b6a7-6695cf0f17e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:12:38.166487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734e6f2-2026-4974-b6a7-6695cf0f17e5 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:12:38.166794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734e6f2-2026-4974-b6a7-6695cf0f17e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:12:38.167072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734e6f2-2026-4974-b6a7-6695cf0f17e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:12:38.176443 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734e6f2-2026-4974-b6a7-6695cf0f17e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:12:38.178084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734e6f2-2026-4974-b6a7-6695cf0f17e5 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:12:38.188204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1205/2396] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:12:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:12:48.184035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=19 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:13:38.052595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fdf9c-4461-44fb-bac8-75060764ce25 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:13:38.054447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f21fdf9c-4461-44fb-bac8-75060764ce25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:13:38.167129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fdf9c-4461-44fb-bac8-75060764ce25 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:13:38.172692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc3927-5a70-46f2-a049-17c495d14394 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:13:38.173360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc3927-5a70-46f2-a049-17c495d14394 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:13:38.173632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc3927-5a70-46f2-a049-17c495d14394 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:13:38.173998 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc3927-5a70-46f2-a049-17c495d14394 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:13:38.176660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d6bc3927-5a70-46f2-a049-17c495d14394 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:13:38.284813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc3927-5a70-46f2-a049-17c495d14394 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:13:38.287480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc3927-5a70-46f2-a049-17c495d14394 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:13:38.312752 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1192/2397] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:13:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:13:40.389339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dea7a-ee32-4d60-9073-81ee7c5deb7c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:13:40.389794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dea7a-ee32-4d60-9073-81ee7c5deb7c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:13:40.389794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dea7a-ee32-4d60-9073-81ee7c5deb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:13:40.482096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f53dea7a-ee32-4d60-9073-81ee7c5deb7c None None] MFA Rules not processed for user `48ff681e601f4892abffe5862def0547`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:13:40.483900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f53dea7a-ee32-4d60-9073-81ee7c5deb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:13:40.507782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1206/2398] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:13:40.744591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc3927-5a70-46f2-a049-17c495d14394 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:13:40.758190 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc3927-5a70-46f2-a049-17c495d14394 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:13:40.766973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836ada4-9ee6-4f3c-bf1f-021494995dfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:13:40.766973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836ada4-9ee6-4f3c-bf1f-021494995dfc service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:13:40.766973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836ada4-9ee6-4f3c-bf1f-021494995dfc service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:13:40.767376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836ada4-9ee6-4f3c-bf1f-021494995dfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:13:40.781394 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836ada4-9ee6-4f3c-bf1f-021494995dfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:13:40.783810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836ada4-9ee6-4f3c-bf1f-021494995dfc service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:13:40.804390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1193/2399] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:13:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:13:41.093736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53dea7a-ee32-4d60-9073-81ee7c5deb7c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:13:41.095369 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f53dea7a-ee32-4d60-9073-81ee7c5deb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:13:41.198149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53dea7a-ee32-4d60-9073-81ee7c5deb7c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:13:41.202429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409fe55-7ee0-4c7e-afbf-eb88c10888ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:13:41.203119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409fe55-7ee0-4c7e-afbf-eb88c10888ab service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:13:41.203396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409fe55-7ee0-4c7e-afbf-eb88c10888ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:13:41.203666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409fe55-7ee0-4c7e-afbf-eb88c10888ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:13:41.206250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b409fe55-7ee0-4c7e-afbf-eb88c10888ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:13:41.218348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409fe55-7ee0-4c7e-afbf-eb88c10888ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:13:41.220179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409fe55-7ee0-4c7e-afbf-eb88c10888ab service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:13:41.232441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1207/2400] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:13:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:13:44.161499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207ab91-dde0-418d-a4e4-de7253e2ba54 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:13:44.162088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207ab91-dde0-418d-a4e4-de7253e2ba54 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:13:44.162088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207ab91-dde0-418d-a4e4-de7253e2ba54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:13:44.204456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-0207ab91-dde0-418d-a4e4-de7253e2ba54 None None] MFA Rules not processed for user `229771b215b3492abc2c5ec38ff0d929`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:13:44.205679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-0207ab91-dde0-418d-a4e4-de7253e2ba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:13:44.226962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:13:44.267182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1194/2401] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:13:45.193372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409fe55-7ee0-4c7e-afbf-eb88c10888ab service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:13:45.210299 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409fe55-7ee0-4c7e-afbf-eb88c10888ab service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:13:45.215743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93350f-0a7b-43de-a24c-4058bc031a3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:13:45.216495 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93350f-0a7b-43de-a24c-4058bc031a3b service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:13:45.216842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93350f-0a7b-43de-a24c-4058bc031a3b service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:13:45.217185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93350f-0a7b-43de-a24c-4058bc031a3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:13:45.219089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ae93350f-0a7b-43de-a24c-4058bc031a3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:13:45.231828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93350f-0a7b-43de-a24c-4058bc031a3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:13:45.233435 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93350f-0a7b-43de-a24c-4058bc031a3b service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:13:45.243854 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1208/2402] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:13:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:13:53.269779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207ab91-dde0-418d-a4e4-de7253e2ba54 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:13:53.290698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207ab91-dde0-418d-a4e4-de7253e2ba54 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:13:53.297590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9caf4-dc1b-4b88-9a15-6e48a824863f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:13:53.298331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9caf4-dc1b-4b88-9a15-6e48a824863f service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:13:53.298709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9caf4-dc1b-4b88-9a15-6e48a824863f service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:13:53.299044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9caf4-dc1b-4b88-9a15-6e48a824863f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:13:53.304821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-03e9caf4-dc1b-4b88-9a15-6e48a824863f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:13:53.318156 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9caf4-dc1b-4b88-9a15-6e48a824863f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:13:53.319845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9caf4-dc1b-4b88-9a15-6e48a824863f service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:13:53.334422 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1195/2403] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:13:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:13:55.239677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=52 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:14:03.329047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=19 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:14:37.664349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93350f-0a7b-43de-a24c-4058bc031a3b service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:14:37.683250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93350f-0a7b-43de-a24c-4058bc031a3b service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:14:37.690591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee783141-498f-4a0d-b09a-0c7721545918 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:14:37.691228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee783141-498f-4a0d-b09a-0c7721545918 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:14:37.691539 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee783141-498f-4a0d-b09a-0c7721545918 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:14:37.691809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee783141-498f-4a0d-b09a-0c7721545918 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:14:37.694040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ee783141-498f-4a0d-b09a-0c7721545918 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:14:37.773740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee783141-498f-4a0d-b09a-0c7721545918 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:14:37.775996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee783141-498f-4a0d-b09a-0c7721545918 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:14:37.790321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1209/2404] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:14:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:14:38.007957 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9caf4-dc1b-4b88-9a15-6e48a824863f service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:14:38.009775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-03e9caf4-dc1b-4b88-9a15-6e48a824863f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:14:38.097740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9caf4-dc1b-4b88-9a15-6e48a824863f service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:14:38.101917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a754d-3670-4646-8155-96cce23cf6be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:14:38.102630 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a754d-3670-4646-8155-96cce23cf6be service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:14:38.102913 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a754d-3670-4646-8155-96cce23cf6be service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:14:38.103189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a754d-3670-4646-8155-96cce23cf6be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:14:38.105862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6b2a754d-3670-4646-8155-96cce23cf6be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:14:38.178704 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a754d-3670-4646-8155-96cce23cf6be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:14:38.180158 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a754d-3670-4646-8155-96cce23cf6be service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:14:38.190002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1196/2405] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:14:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:14:38.255191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee783141-498f-4a0d-b09a-0c7721545918 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:14:38.265740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee783141-498f-4a0d-b09a-0c7721545918 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:14:38.269919 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520a17a-bb35-49b0-904d-79e2579e29ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:14:38.270588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520a17a-bb35-49b0-904d-79e2579e29ff service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:14:38.270887 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520a17a-bb35-49b0-904d-79e2579e29ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:14:38.271160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520a17a-bb35-49b0-904d-79e2579e29ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:14:38.273247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9520a17a-bb35-49b0-904d-79e2579e29ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:14:38.290249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520a17a-bb35-49b0-904d-79e2579e29ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:14:38.292134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520a17a-bb35-49b0-904d-79e2579e29ff service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:14:38.302446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1210/2406] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:14:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:14:41.467599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a754d-3670-4646-8155-96cce23cf6be service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:14:41.469674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6b2a754d-3670-4646-8155-96cce23cf6be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:14:41.483101 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a754d-3670-4646-8155-96cce23cf6be service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:14:41.487547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408704a2-3ebe-4725-9cb6-3b8fa5985b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:14:41.488274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408704a2-3ebe-4725-9cb6-3b8fa5985b53 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:14:41.488477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408704a2-3ebe-4725-9cb6-3b8fa5985b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:14:41.488767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408704a2-3ebe-4725-9cb6-3b8fa5985b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:14:41.490907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-408704a2-3ebe-4725-9cb6-3b8fa5985b53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:14:41.501488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408704a2-3ebe-4725-9cb6-3b8fa5985b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:14:41.502993 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408704a2-3ebe-4725-9cb6-3b8fa5985b53 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:14:41.512998 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1197/2407] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:14:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:14:48.297364 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=39 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:14:51.507724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=68 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:16:35.999236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520a17a-bb35-49b0-904d-79e2579e29ff service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:16:36.000992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9520a17a-bb35-49b0-904d-79e2579e29ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:16:36.013450 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520a17a-bb35-49b0-904d-79e2579e29ff service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:16:36.018312 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fc9ca-28dd-42c0-a652-78eb56be6b26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:16:36.018999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc9ca-28dd-42c0-a652-78eb56be6b26 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:16:36.019294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc9ca-28dd-42c0-a652-78eb56be6b26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:16:36.019566 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc9ca-28dd-42c0-a652-78eb56be6b26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:16:36.021511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f21fc9ca-28dd-42c0-a652-78eb56be6b26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:16:36.099702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fc9ca-28dd-42c0-a652-78eb56be6b26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:16:36.101180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fc9ca-28dd-42c0-a652-78eb56be6b26 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:16:36.111034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1211/2408] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:16:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:16:37.668216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408704a2-3ebe-4725-9cb6-3b8fa5985b53 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:16:37.682200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408704a2-3ebe-4725-9cb6-3b8fa5985b53 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:16:37.687253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c08c17-7372-4f8c-adfa-af507ca1eb5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:16:37.687836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c08c17-7372-4f8c-adfa-af507ca1eb5f service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:16:37.688121 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c08c17-7372-4f8c-adfa-af507ca1eb5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:16:37.688413 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c08c17-7372-4f8c-adfa-af507ca1eb5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:16:37.690716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-05c08c17-7372-4f8c-adfa-af507ca1eb5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:16:37.760317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c08c17-7372-4f8c-adfa-af507ca1eb5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:16:37.761896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c08c17-7372-4f8c-adfa-af507ca1eb5f service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:16:37.771573 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1198/2409] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:16:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:16:46.106452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=33 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:16:47.767085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=28 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:17:42.129823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fc9ca-28dd-42c0-a652-78eb56be6b26 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:17:42.131583 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f21fc9ca-28dd-42c0-a652-78eb56be6b26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:17:42.204230 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fc9ca-28dd-42c0-a652-78eb56be6b26 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:17:42.210688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7e7c7-df23-4756-a6f6-0eee76b5a396 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:17:42.211345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7e7c7-df23-4756-a6f6-0eee76b5a396 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:17:42.211621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7e7c7-df23-4756-a6f6-0eee76b5a396 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:17:42.211888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7e7c7-df23-4756-a6f6-0eee76b5a396 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:17:42.213926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a7f7e7c7-df23-4756-a6f6-0eee76b5a396 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:17:42.226806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7e7c7-df23-4756-a6f6-0eee76b5a396 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:17:42.228415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7e7c7-df23-4756-a6f6-0eee76b5a396 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:17:42.238939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1212/2410] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:17:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:17:52.234686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=32 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:18:39.092821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c08c17-7372-4f8c-adfa-af507ca1eb5f service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:18:39.102604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c08c17-7372-4f8c-adfa-af507ca1eb5f service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:18:39.107308 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9f4f6-1d50-461d-bb2c-b174ada6a275 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:18:39.107983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9f4f6-1d50-461d-bb2c-b174ada6a275 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:18:39.108257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9f4f6-1d50-461d-bb2c-b174ada6a275 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:18:39.108529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9f4f6-1d50-461d-bb2c-b174ada6a275 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:18:39.119630 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9f4f6-1d50-461d-bb2c-b174ada6a275 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:18:39.121100 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9f4f6-1d50-461d-bb2c-b174ada6a275 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:18:39.132030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1199/2411] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:18:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:18:49.126840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=6 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:19:38.016722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7e7c7-df23-4756-a6f6-0eee76b5a396 service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:38.031665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7e7c7-df23-4756-a6f6-0eee76b5a396 service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:38.038976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e3578-28ed-478c-882e-0da0a2166704 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:38.039561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e3578-28ed-478c-882e-0da0a2166704 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:38.039836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e3578-28ed-478c-882e-0da0a2166704 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:38.040106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e3578-28ed-478c-882e-0da0a2166704 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:38.049849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e3578-28ed-478c-882e-0da0a2166704 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:38.051480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e3578-28ed-478c-882e-0da0a2166704 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:38.082011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1213/2412] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:19:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:19:38.694043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9f4f6-1d50-461d-bb2c-b174ada6a275 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:38.704047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9f4f6-1d50-461d-bb2c-b174ada6a275 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:38.709053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0420dab-52bf-421f-ac15-d5d8611a16c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:38.711673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0420dab-52bf-421f-ac15-d5d8611a16c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:38.711959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0420dab-52bf-421f-ac15-d5d8611a16c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:38.712242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0420dab-52bf-421f-ac15-d5d8611a16c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:38.722390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0420dab-52bf-421f-ac15-d5d8611a16c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:38.723882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0420dab-52bf-421f-ac15-d5d8611a16c6 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:38.753518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1200/2413] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:19:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:19:38.806271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e3578-28ed-478c-882e-0da0a2166704 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:38.817714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e3578-28ed-478c-882e-0da0a2166704 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:38.823238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf83cbb-1bcf-4c89-8559-6dacf2b22ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:38.823883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf83cbb-1bcf-4c89-8559-6dacf2b22ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:38.824159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf83cbb-1bcf-4c89-8559-6dacf2b22ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:38.824425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf83cbb-1bcf-4c89-8559-6dacf2b22ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:38.834962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf83cbb-1bcf-4c89-8559-6dacf2b22ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:38.836402 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf83cbb-1bcf-4c89-8559-6dacf2b22ef6 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:38.868050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1214/2414] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:19:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:19:40.209661 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0420dab-52bf-421f-ac15-d5d8611a16c6 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:40.220203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0420dab-52bf-421f-ac15-d5d8611a16c6 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:40.223885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53b19f-fb95-49f0-abae-82cc2a8fcc78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:40.224535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53b19f-fb95-49f0-abae-82cc2a8fcc78 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:40.224814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53b19f-fb95-49f0-abae-82cc2a8fcc78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:40.225089 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53b19f-fb95-49f0-abae-82cc2a8fcc78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:40.227561 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5d53b19f-fb95-49f0-abae-82cc2a8fcc78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:19:40.242239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53b19f-fb95-49f0-abae-82cc2a8fcc78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:40.243745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53b19f-fb95-49f0-abae-82cc2a8fcc78 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:40.254936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1201/2415] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:19:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:19:41.445954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf83cbb-1bcf-4c89-8559-6dacf2b22ef6 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:41.455838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf83cbb-1bcf-4c89-8559-6dacf2b22ef6 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:41.459530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba934e9-10dc-48b2-ac90-155eac8a6775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:41.460178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba934e9-10dc-48b2-ac90-155eac8a6775 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:41.460460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba934e9-10dc-48b2-ac90-155eac8a6775 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:41.460863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba934e9-10dc-48b2-ac90-155eac8a6775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:41.462798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1ba934e9-10dc-48b2-ac90-155eac8a6775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:19:41.477686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba934e9-10dc-48b2-ac90-155eac8a6775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:41.479184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba934e9-10dc-48b2-ac90-155eac8a6775 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:41.491264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1215/2416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:19:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:19:43.793513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53b19f-fb95-49f0-abae-82cc2a8fcc78 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:43.820639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53b19f-fb95-49f0-abae-82cc2a8fcc78 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:43.831886 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a2ac7-efb6-4bee-a3d9-aedaaf8c6560 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:43.832950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a2ac7-efb6-4bee-a3d9-aedaaf8c6560 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:43.833702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a2ac7-efb6-4bee-a3d9-aedaaf8c6560 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:43.834943 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a2ac7-efb6-4bee-a3d9-aedaaf8c6560 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:43.839781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ba6a2ac7-efb6-4bee-a3d9-aedaaf8c6560 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:19:43.877615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a2ac7-efb6-4bee-a3d9-aedaaf8c6560 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:43.880287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a2ac7-efb6-4bee-a3d9-aedaaf8c6560 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:43.903651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1202/2417] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:19:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:19:43.935694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba934e9-10dc-48b2-ac90-155eac8a6775 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:43.948573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba934e9-10dc-48b2-ac90-155eac8a6775 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:43.954837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6aaf1-f015-4dd3-94a3-11a211213e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:43.955516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6aaf1-f015-4dd3-94a3-11a211213e12 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:43.955800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6aaf1-f015-4dd3-94a3-11a211213e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:43.956118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6aaf1-f015-4dd3-94a3-11a211213e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:43.958914 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6ac6aaf1-f015-4dd3-94a3-11a211213e12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:19:44.085580 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6aaf1-f015-4dd3-94a3-11a211213e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:44.087182 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6aaf1-f015-4dd3-94a3-11a211213e12 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:44.113123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1216/2418] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:19:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:19:45.829699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a2ac7-efb6-4bee-a3d9-aedaaf8c6560 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:45.848501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a2ac7-efb6-4bee-a3d9-aedaaf8c6560 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:45.853509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb519c9-5b12-4e0e-a64b-f90f9835d108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:45.854307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb519c9-5b12-4e0e-a64b-f90f9835d108 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:45.854659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb519c9-5b12-4e0e-a64b-f90f9835d108 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:45.855209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb519c9-5b12-4e0e-a64b-f90f9835d108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:45.866836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb519c9-5b12-4e0e-a64b-f90f9835d108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:45.868846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb519c9-5b12-4e0e-a64b-f90f9835d108 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:45.879428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1203/2419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:19:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:19:48.062584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6aaf1-f015-4dd3-94a3-11a211213e12 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:48.075453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6aaf1-f015-4dd3-94a3-11a211213e12 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:48.080587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c784e8c-d499-4d71-b596-de9f2f2ff568 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:48.081246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c784e8c-d499-4d71-b596-de9f2f2ff568 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:48.081530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c784e8c-d499-4d71-b596-de9f2f2ff568 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:48.081805 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c784e8c-d499-4d71-b596-de9f2f2ff568 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:48.083591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7c784e8c-d499-4d71-b596-de9f2f2ff568 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:19:48.095906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c784e8c-d499-4d71-b596-de9f2f2ff568 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:48.097510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c784e8c-d499-4d71-b596-de9f2f2ff568 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:48.108031 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1217/2420] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:19:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:19:52.130743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb519c9-5b12-4e0e-a64b-f90f9835d108 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:52.132534 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4fb519c9-5b12-4e0e-a64b-f90f9835d108 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:19:52.156708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb519c9-5b12-4e0e-a64b-f90f9835d108 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:52.162666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:52.163553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:52.163835 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:52.164111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] PATH_INFO: `/v3/users/48ff681e601f4892abffe5862def0547` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:52.165449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ff681e601f4892abffe5862def0547)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:52.167447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:52.220620 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ff681e601f4892abffe5862def0547'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.234550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ff681e601f4892abffe5862def0547'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.235557 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ff681e601f4892abffe5862def0547'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.236379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ff681e601f4892abffe5862def0547'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.245706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ff681e601f4892abffe5862def0547'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.261394 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1204/2421] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:19:52 2022] DELETE /identity/v3/users/48ff681e601f4892abffe5862def0547 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:19:52.450691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c784e8c-d499-4d71-b596-de9f2f2ff568 service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:52.461855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c784e8c-d499-4d71-b596-de9f2f2ff568 service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:52.506199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:19:52.524350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:52.525215 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:52.525482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:52.525754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] PATH_INFO: `/v3/projects/0192e4102b644993a9482674674b1ad7` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:52.526998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0192e4102b644993a9482674674b1ad7)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:52.529458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:52.577998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0192e4102b644993a9482674674b1ad7', 'request_id': 'req-ecee7b08-18df-49c7-82e4-64daa577e06d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.586733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0192e4102b644993a9482674674b1ad7', 'request_id': 'req-ecee7b08-18df-49c7-82e4-64daa577e06d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.596377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0192e4102b644993a9482674674b1ad7', 'request_id': 'req-ecee7b08-18df-49c7-82e4-64daa577e06d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.617059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] The token cache is being invalidate because project 0192e4102b644993a9482674674b1ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:19:52.617525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.618465 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.620883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1218/2422] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:19:52 2022] DELETE /identity/v3/projects/0192e4102b644993a9482674674b1ad7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:19:52.627400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:52.629476 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:19:52.649081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10f972-cd60-40e2-8203-48aa21e60b15 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:52.722724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:52.723652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:52.723927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:52.724194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] PATH_INFO: `/v3/users/5a2d88a23143416083a691af22d70a66` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:52.733914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:19:52.742928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2d88a23143416083a691af22d70a66)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:52.753529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:52.799683 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2d88a23143416083a691af22d70a66'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.812381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2d88a23143416083a691af22d70a66'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.813120 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2d88a23143416083a691af22d70a66'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.813711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2d88a23143416083a691af22d70a66'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.821026 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2d88a23143416083a691af22d70a66'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:52.834269 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1205/2423] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:19:52 2022] DELETE /identity/v3/users/5a2d88a23143416083a691af22d70a66 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:19:53.020827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:19:53.022317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:19:53.036426 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7b08-18df-49c7-82e4-64daa577e06d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:19:53.103155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:19:53.103909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:19:53.104193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:19:53.104454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] PATH_INFO: `/v3/projects/eb2108eb54074b92896b5f0fe77ecf3f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:19:53.118513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb2108eb54074b92896b5f0fe77ecf3f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:19:53.120899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:19:53.168758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb2108eb54074b92896b5f0fe77ecf3f', 'request_id': 'req-f719acff-bfa3-4d41-97cd-51eede2963ed'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:53.177282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb2108eb54074b92896b5f0fe77ecf3f', 'request_id': 'req-f719acff-bfa3-4d41-97cd-51eede2963ed'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:53.183945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb2108eb54074b92896b5f0fe77ecf3f', 'request_id': 'req-f719acff-bfa3-4d41-97cd-51eede2963ed'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:53.185178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:19:53.199164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] The token cache is being invalidate because project eb2108eb54074b92896b5f0fe77ecf3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:19:53.199497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:53.200421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:19:53.201972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1219/2424] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:19:53 2022] DELETE /identity/v3/projects/eb2108eb54074b92896b5f0fe77ecf3f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:20:02.830923 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:20:03.197929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:21:37.091044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:21:37.092710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:21:37.168104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9969f-0076-4414-82df-a024510dbdd9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:21:37.244555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ee106-e58c-49aa-9388-ccc96866c05a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:21:37.245219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee106-e58c-49aa-9388-ccc96866c05a service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:21:37.245472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee106-e58c-49aa-9388-ccc96866c05a service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:21:37.245741 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee106-e58c-49aa-9388-ccc96866c05a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:21:37.247982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-764ee106-e58c-49aa-9388-ccc96866c05a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:21:37.319891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ee106-e58c-49aa-9388-ccc96866c05a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:21:37.322630 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ee106-e58c-49aa-9388-ccc96866c05a service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:21:37.338135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1206/2425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:21:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:21:47.331174 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=84 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:22:48.197623 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:22:48.199626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:22:48.361135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719acff-bfa3-4d41-97cd-51eede2963ed admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:22:48.367224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f89bc-f0f0-47e5-a997-0c1ff556fe73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:22:48.367880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f89bc-f0f0-47e5-a997-0c1ff556fe73 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:22:48.368157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f89bc-f0f0-47e5-a997-0c1ff556fe73 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:22:48.368429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f89bc-f0f0-47e5-a997-0c1ff556fe73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:22:48.370535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-df1f89bc-f0f0-47e5-a997-0c1ff556fe73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:22:48.383097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f89bc-f0f0-47e5-a997-0c1ff556fe73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:22:48.384733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f89bc-f0f0-47e5-a997-0c1ff556fe73 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:22:48.395767 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1220/2426] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:22:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:22:58.390853 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=63 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:23:38.731837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ee106-e58c-49aa-9388-ccc96866c05a service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:23:38.735097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-764ee106-e58c-49aa-9388-ccc96866c05a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:23:38.839187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ee106-e58c-49aa-9388-ccc96866c05a service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:23:38.845787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ae689-d634-4a69-8fc0-84d4768b4838 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:23:38.846443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ae689-d634-4a69-8fc0-84d4768b4838 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:23:38.846747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ae689-d634-4a69-8fc0-84d4768b4838 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:23:38.847020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ae689-d634-4a69-8fc0-84d4768b4838 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:23:38.848925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-723ae689-d634-4a69-8fc0-84d4768b4838 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:23:38.942265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ae689-d634-4a69-8fc0-84d4768b4838 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:23:38.943934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ae689-d634-4a69-8fc0-84d4768b4838 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:23:38.964264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1207/2427] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:23:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:23:48.961483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=75 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:24:38.912545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f89bc-f0f0-47e5-a997-0c1ff556fe73 service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:24:38.914247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-df1f89bc-f0f0-47e5-a997-0c1ff556fe73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:24:38.930382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f89bc-f0f0-47e5-a997-0c1ff556fe73 service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:24:38.936669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508e521-0966-4a42-8613-a004eecbf2d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:24:38.937316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508e521-0966-4a42-8613-a004eecbf2d3 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:24:38.937596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508e521-0966-4a42-8613-a004eecbf2d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:24:38.937858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508e521-0966-4a42-8613-a004eecbf2d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:24:38.940338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7508e521-0966-4a42-8613-a004eecbf2d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:24:39.016213 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508e521-0966-4a42-8613-a004eecbf2d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:24:39.017829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508e521-0966-4a42-8613-a004eecbf2d3 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:24:39.031254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1221/2428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:24:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:24:39.560714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ae689-d634-4a69-8fc0-84d4768b4838 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:24:39.563157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-723ae689-d634-4a69-8fc0-84d4768b4838 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:24:39.653366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ae689-d634-4a69-8fc0-84d4768b4838 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:24:39.657496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ea892-81b6-4884-ba39-1fdc6adc7983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:24:39.658145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea892-81b6-4884-ba39-1fdc6adc7983 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:24:39.658419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea892-81b6-4884-ba39-1fdc6adc7983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:24:39.658691 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea892-81b6-4884-ba39-1fdc6adc7983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:24:39.660669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-079ea892-81b6-4884-ba39-1fdc6adc7983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:24:39.677391 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ea892-81b6-4884-ba39-1fdc6adc7983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:24:39.679166 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ea892-81b6-4884-ba39-1fdc6adc7983 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:24:39.690985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1208/2429] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:24:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:24:40.202844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508e521-0966-4a42-8613-a004eecbf2d3 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:24:40.205243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7508e521-0966-4a42-8613-a004eecbf2d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:24:40.226254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508e521-0966-4a42-8613-a004eecbf2d3 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:24:40.230458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f808f3-e36a-4857-9d09-4a954cf8c0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:24:40.231184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f808f3-e36a-4857-9d09-4a954cf8c0ba service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:24:40.231470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f808f3-e36a-4857-9d09-4a954cf8c0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:24:40.231747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f808f3-e36a-4857-9d09-4a954cf8c0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:24:40.233726 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-26f808f3-e36a-4857-9d09-4a954cf8c0ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:24:40.248121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f808f3-e36a-4857-9d09-4a954cf8c0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:24:40.249558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f808f3-e36a-4857-9d09-4a954cf8c0ba service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:24:40.260684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1222/2430] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:24:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:24:49.685653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=35 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:24:50.255743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=39 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:24:52.408992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ea892-81b6-4884-ba39-1fdc6adc7983 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:24:52.418527 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ea892-81b6-4884-ba39-1fdc6adc7983 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:24:52.422909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3bae2-925e-4afa-bb71-1d0dea77576b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:24:52.423562 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3bae2-925e-4afa-bb71-1d0dea77576b service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:24:52.423840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3bae2-925e-4afa-bb71-1d0dea77576b service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:24:52.424115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3bae2-925e-4afa-bb71-1d0dea77576b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:24:52.426826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-67e3bae2-925e-4afa-bb71-1d0dea77576b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:24:52.442826 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3bae2-925e-4afa-bb71-1d0dea77576b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:24:52.444359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3bae2-925e-4afa-bb71-1d0dea77576b service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:24:52.454295 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1209/2431] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:24:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:25:02.449635 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=6 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:25:40.774675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62ac05-1904-4f97-8507-d4a491cc1dad None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:40.775166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62ac05-1904-4f97-8507-d4a491cc1dad None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:40.775284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62ac05-1904-4f97-8507-d4a491cc1dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:40.841165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ea62ac05-1904-4f97-8507-d4a491cc1dad None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:25:40.843001 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ea62ac05-1904-4f97-8507-d4a491cc1dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:40.920942 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1223/2432] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jun 29 10:25:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:25:40.973514 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3bae2-925e-4afa-bb71-1d0dea77576b service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:40.975709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-67e3bae2-925e-4afa-bb71-1d0dea77576b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:41.073735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3bae2-925e-4afa-bb71-1d0dea77576b service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:41.079965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286fc1a-0ea6-4f40-bba2-9bcfede3bf33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:41.080612 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286fc1a-0ea6-4f40-bba2-9bcfede3bf33 service swift] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:41.080890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286fc1a-0ea6-4f40-bba2-9bcfede3bf33 service swift] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:41.081158 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286fc1a-0ea6-4f40-bba2-9bcfede3bf33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:41.090489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286fc1a-0ea6-4f40-bba2-9bcfede3bf33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:41.092114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286fc1a-0ea6-4f40-bba2-9bcfede3bf33 service swift] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:41.100813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1210/2433] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:25:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:25:41.801711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62ac05-1904-4f97-8507-d4a491cc1dad None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:41.811907 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62ac05-1904-4f97-8507-d4a491cc1dad None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:41.816030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c42c8f-fd44-4426-95cb-86d150639a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:41.816692 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c42c8f-fd44-4426-95cb-86d150639a40 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:41.817051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c42c8f-fd44-4426-95cb-86d150639a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:41.817350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c42c8f-fd44-4426-95cb-86d150639a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:41.819319 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e5c42c8f-fd44-4426-95cb-86d150639a40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:41.901492 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c42c8f-fd44-4426-95cb-86d150639a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:41.903002 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c42c8f-fd44-4426-95cb-86d150639a40 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:41.916070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1224/2434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:25:44.175013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286fc1a-0ea6-4f40-bba2-9bcfede3bf33 service swift] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:44.176558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9286fc1a-0ea6-4f40-bba2-9bcfede3bf33 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:44.282045 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286fc1a-0ea6-4f40-bba2-9bcfede3bf33 service swift] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:44.285755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca10fa-b9d3-47ea-98a0-5f51adad4ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:44.286412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca10fa-b9d3-47ea-98a0-5f51adad4ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:44.286738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca10fa-b9d3-47ea-98a0-5f51adad4ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:44.287011 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca10fa-b9d3-47ea-98a0-5f51adad4ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:44.289270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-03ca10fa-b9d3-47ea-98a0-5f51adad4ae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:44.372523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:25:44.382376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca10fa-b9d3-47ea-98a0-5f51adad4ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:44.383895 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca10fa-b9d3-47ea-98a0-5f51adad4ae6 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:44.394331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1211/2435] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:25:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:25:44.446981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c42c8f-fd44-4426-95cb-86d150639a40 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:44.460074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c42c8f-fd44-4426-95cb-86d150639a40 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:44.464341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f008fa-330e-413f-a67c-857f1512b4fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:44.465024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f008fa-330e-413f-a67c-857f1512b4fe service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:44.465507 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f008fa-330e-413f-a67c-857f1512b4fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:44.465840 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f008fa-330e-413f-a67c-857f1512b4fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:44.467688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f8f008fa-330e-413f-a67c-857f1512b4fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:44.555713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:25:44.571895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f008fa-330e-413f-a67c-857f1512b4fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:44.573380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f008fa-330e-413f-a67c-857f1512b4fe service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:44.582793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1225/2436] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:25:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:25:45.399531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168abb7-354e-4585-b01a-e6c4eeb77857 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:45.400111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168abb7-354e-4585-b01a-e6c4eeb77857 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:45.400446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168abb7-354e-4585-b01a-e6c4eeb77857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:45.433894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-7168abb7-354e-4585-b01a-e6c4eeb77857 None None] MFA Rules not processed for user `7900ba750c5f4e9faa69baaed1fd95d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:25:45.435639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7168abb7-354e-4585-b01a-e6c4eeb77857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:45.445959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1212/2437] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:25:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:25:45.620613 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f008fa-330e-413f-a67c-857f1512b4fe service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:45.622129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f8f008fa-330e-413f-a67c-857f1512b4fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:45.638933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f008fa-330e-413f-a67c-857f1512b4fe service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:45.644179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e269e5e-78b0-4100-aba8-d10c302474d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:45.644834 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e269e5e-78b0-4100-aba8-d10c302474d4 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:45.645116 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e269e5e-78b0-4100-aba8-d10c302474d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:45.645388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e269e5e-78b0-4100-aba8-d10c302474d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:45.647990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2e269e5e-78b0-4100-aba8-d10c302474d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:45.660723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e269e5e-78b0-4100-aba8-d10c302474d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:45.662688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e269e5e-78b0-4100-aba8-d10c302474d4 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:45.676579 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1226/2438] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:25:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:25:45.890916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168abb7-354e-4585-b01a-e6c4eeb77857 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:45.903300 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168abb7-354e-4585-b01a-e6c4eeb77857 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:45.908362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0ce57-8b82-43fe-807f-edbe09907afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:45.909291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0ce57-8b82-43fe-807f-edbe09907afc service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:45.909617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0ce57-8b82-43fe-807f-edbe09907afc service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:45.909950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0ce57-8b82-43fe-807f-edbe09907afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:45.912462 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b5b0ce57-8b82-43fe-807f-edbe09907afc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:45.923360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0ce57-8b82-43fe-807f-edbe09907afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:45.924804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0ce57-8b82-43fe-807f-edbe09907afc service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:45.934463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1213/2439] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:25:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:25:46.433903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e269e5e-78b0-4100-aba8-d10c302474d4 service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:46.447262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e269e5e-78b0-4100-aba8-d10c302474d4 service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:46.452263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb09a4-ea5d-4f9c-81e5-7d0553c19b51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:46.452913 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb09a4-ea5d-4f9c-81e5-7d0553c19b51 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:46.453226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb09a4-ea5d-4f9c-81e5-7d0553c19b51 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:46.453519 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb09a4-ea5d-4f9c-81e5-7d0553c19b51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:46.455257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4fdb09a4-ea5d-4f9c-81e5-7d0553c19b51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:46.467126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb09a4-ea5d-4f9c-81e5-7d0553c19b51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:46.468723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb09a4-ea5d-4f9c-81e5-7d0553c19b51 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:46.478849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1227/2440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:25:50.028090 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0ce57-8b82-43fe-807f-edbe09907afc service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:50.060496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0ce57-8b82-43fe-807f-edbe09907afc service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:50.067593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a861f-01df-4136-8838-acf6ad506127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:50.068493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a861f-01df-4136-8838-acf6ad506127 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:50.068747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a861f-01df-4136-8838-acf6ad506127 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:50.069414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a861f-01df-4136-8838-acf6ad506127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:50.072595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cb7a861f-01df-4136-8838-acf6ad506127 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:50.092904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a861f-01df-4136-8838-acf6ad506127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:50.095131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a861f-01df-4136-8838-acf6ad506127 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:50.122429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1214/2441] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:25:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:25:50.202277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb09a4-ea5d-4f9c-81e5-7d0553c19b51 service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:50.203762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4fdb09a4-ea5d-4f9c-81e5-7d0553c19b51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:50.227966 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb09a4-ea5d-4f9c-81e5-7d0553c19b51 service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:50.235349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:50.237268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:50.242104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:50.243197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] PATH_INFO: `/v3/users/f2ed196e5ad6415aa9b7a7c1b3dad277` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:50.245653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2ed196e5ad6415aa9b7a7c1b3dad277)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:50.247620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:50.322948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2ed196e5ad6415aa9b7a7c1b3dad277'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:50.335530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2ed196e5ad6415aa9b7a7c1b3dad277'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:50.336285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2ed196e5ad6415aa9b7a7c1b3dad277'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:50.343186 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2ed196e5ad6415aa9b7a7c1b3dad277'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:50.343883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2ed196e5ad6415aa9b7a7c1b3dad277'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:50.360521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1228/2442] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:25:50 2022] DELETE /identity/v3/users/f2ed196e5ad6415aa9b7a7c1b3dad277 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:25:50.793886 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a861f-01df-4136-8838-acf6ad506127 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:50.805334 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a861f-01df-4136-8838-acf6ad506127 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:50.864658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:50.865853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:50.866180 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:50.866451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] PATH_INFO: `/v3/projects/ce73b243663d4ee6b991c9262d1286d5` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:50.877669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce73b243663d4ee6b991c9262d1286d5)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:50.879661 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:50.926657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce73b243663d4ee6b991c9262d1286d5', 'request_id': 'req-d47269d9-7e78-4ec2-9610-c3198d61ee99'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:50.935875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce73b243663d4ee6b991c9262d1286d5', 'request_id': 'req-d47269d9-7e78-4ec2-9610-c3198d61ee99'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:50.942046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce73b243663d4ee6b991c9262d1286d5', 'request_id': 'req-d47269d9-7e78-4ec2-9610-c3198d61ee99'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:50.953614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] The token cache is being invalidate because project ce73b243663d4ee6b991c9262d1286d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:25:50.953896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:50.954514 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:50.956071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1215/2443] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:25:50 2022] DELETE /identity/v3/projects/ce73b243663d4ee6b991c9262d1286d5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:25:50.961092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:50.962716 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:50.975614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff2cea-29f0-4631-9f34-78e7b9c95130 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:51.042883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:51.050427 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:51.050427 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:51.050427 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] PATH_INFO: `/v3/users/ea0a851d0a1844e2b9f2f949898b7190` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:51.060316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea0a851d0a1844e2b9f2f949898b7190)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:51.062304 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:51.084689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:25:51.113935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea0a851d0a1844e2b9f2f949898b7190'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:51.125631 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea0a851d0a1844e2b9f2f949898b7190'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:51.126288 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea0a851d0a1844e2b9f2f949898b7190'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:51.132976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea0a851d0a1844e2b9f2f949898b7190'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:51.133632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea0a851d0a1844e2b9f2f949898b7190'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:51.147499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1229/2444] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:25:50 2022] DELETE /identity/v3/users/ea0a851d0a1844e2b9f2f949898b7190 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:25:51.336638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:51.338197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:51.352687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47269d9-7e78-4ec2-9610-c3198d61ee99 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:51.406189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:25:51.442080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:51.443003 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:51.443294 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:51.443642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] PATH_INFO: `/v3/projects/caccf28b6b80494393ac62c201fce18e` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:51.462050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] RBAC: Authorizing `identity:delete_project(project_id=caccf28b6b80494393ac62c201fce18e)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:51.464035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:51.514004 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caccf28b6b80494393ac62c201fce18e', 'request_id': 'req-a007149e-0d9b-42ba-91b6-78071286206b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:51.523138 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caccf28b6b80494393ac62c201fce18e', 'request_id': 'req-a007149e-0d9b-42ba-91b6-78071286206b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:51.529978 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caccf28b6b80494393ac62c201fce18e', 'request_id': 'req-a007149e-0d9b-42ba-91b6-78071286206b'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:51.542002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] The token cache is being invalidate because project caccf28b6b80494393ac62c201fce18e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:25:51.542293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:51.543013 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:25:51.544860 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1216/2445] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:25:51 2022] DELETE /identity/v3/projects/caccf28b6b80494393ac62c201fce18e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:25:55.453948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:25:55.455515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:55.539030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e14d0f-93be-40a7-b22f-c8b7f72537f2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:25:55.607916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b6c0a-2cc5-4f36-8186-226dc4893f61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:25:55.608572 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b6c0a-2cc5-4f36-8186-226dc4893f61 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:25:55.608852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b6c0a-2cc5-4f36-8186-226dc4893f61 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:25:55.609125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b6c0a-2cc5-4f36-8186-226dc4893f61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:25:55.611140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-511b6c0a-2cc5-4f36-8186-226dc4893f61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:25:55.624683 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b6c0a-2cc5-4f36-8186-226dc4893f61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:25:55.626287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b6c0a-2cc5-4f36-8186-226dc4893f61 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:25:55.637321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1230/2446] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:25:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:26:01.539289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:26:05.632609 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=7,keystone:SELECT=82,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:27:39.774789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:27:39.776321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:27:39.842021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007149e-0d9b-42ba-91b6-78071286206b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:27:39.846280 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048a683-bc7c-4b5c-9e44-67473252f45a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:27:39.846992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048a683-bc7c-4b5c-9e44-67473252f45a service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:27:39.847276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048a683-bc7c-4b5c-9e44-67473252f45a service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:27:39.847547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048a683-bc7c-4b5c-9e44-67473252f45a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:27:39.849692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4048a683-bc7c-4b5c-9e44-67473252f45a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application [None req-4048a683-bc7c-4b5c-9e44-67473252f45a service cinder] Could not find project: ce73b243663d4ee6b991c9262d1286d5.: keystone.exception.ProjectNotFound: Could not find project: ce73b243663d4ee6b991c9262d1286d5. Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jun 29 10:27:39.889468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 10:27:39.891056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self._validate_token_resources() Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 10:27:39.892455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application with Lock( Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._enter() Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.creator() Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application created_value = creator( Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 10:27:39.894305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 10:27:39.896474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: ce73b243663d4ee6b991c9262d1286d5. Jun 29 10:27:39.896474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: ERROR keystone.server.flask.application Jun 29 10:27:39.896474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1217/2447] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:27:39 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 116 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jun 29 10:27:49.853907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=24 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:27:54.088412 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b6c0a-2cc5-4f36-8186-226dc4893f61 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:27:54.089926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-511b6c0a-2cc5-4f36-8186-226dc4893f61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:27:54.163659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b6c0a-2cc5-4f36-8186-226dc4893f61 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:27:54.169897 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbc082-0525-41ad-9166-a17385363c66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:27:54.170544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbc082-0525-41ad-9166-a17385363c66 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:27:54.170853 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbc082-0525-41ad-9166-a17385363c66 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:27:54.171198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbc082-0525-41ad-9166-a17385363c66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:27:54.173316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-afcbc082-0525-41ad-9166-a17385363c66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:27:54.190606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbc082-0525-41ad-9166-a17385363c66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:27:54.193060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbc082-0525-41ad-9166-a17385363c66 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:27:54.206839 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1231/2448] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:27:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:28:04.201415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=32 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:29:40.539064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048a683-bc7c-4b5c-9e44-67473252f45a service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:29:40.548735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048a683-bc7c-4b5c-9e44-67473252f45a service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:29:40.553071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ebc4c-9005-4a1e-8432-53e7ad2830a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:29:40.553709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebc4c-9005-4a1e-8432-53e7ad2830a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:29:40.553982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebc4c-9005-4a1e-8432-53e7ad2830a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:29:40.554245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebc4c-9005-4a1e-8432-53e7ad2830a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:29:40.556572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-144ebc4c-9005-4a1e-8432-53e7ad2830a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:29:40.620304 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ebc4c-9005-4a1e-8432-53e7ad2830a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:29:40.621757 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ebc4c-9005-4a1e-8432-53e7ad2830a0 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:29:40.630859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1218/2449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:29:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:29:40.657664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbc082-0525-41ad-9166-a17385363c66 service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:29:40.668236 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbc082-0525-41ad-9166-a17385363c66 service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:29:40.672262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc97e9-5a8f-48de-b4b0-b6187039a636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:29:40.672895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc97e9-5a8f-48de-b4b0-b6187039a636 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:29:40.673176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc97e9-5a8f-48de-b4b0-b6187039a636 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:29:40.673442 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc97e9-5a8f-48de-b4b0-b6187039a636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:29:40.675456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-41dc97e9-5a8f-48de-b4b0-b6187039a636 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:29:40.747519 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc97e9-5a8f-48de-b4b0-b6187039a636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:29:40.748865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc97e9-5a8f-48de-b4b0-b6187039a636 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:29:40.768130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1232/2450] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:29:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:29:41.234214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ebc4c-9005-4a1e-8432-53e7ad2830a0 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:29:41.235784 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-144ebc4c-9005-4a1e-8432-53e7ad2830a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:29:41.301530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ebc4c-9005-4a1e-8432-53e7ad2830a0 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:29:41.306135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b72d7b-7394-4e03-99d5-3eb34addc70b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:29:41.306734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b72d7b-7394-4e03-99d5-3eb34addc70b service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:29:41.307017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b72d7b-7394-4e03-99d5-3eb34addc70b service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:29:41.307293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b72d7b-7394-4e03-99d5-3eb34addc70b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:29:41.309310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c3b72d7b-7394-4e03-99d5-3eb34addc70b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:29:41.323978 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b72d7b-7394-4e03-99d5-3eb34addc70b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:29:41.325441 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b72d7b-7394-4e03-99d5-3eb34addc70b service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:29:41.336603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1219/2451] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:29:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:29:42.512433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc97e9-5a8f-48de-b4b0-b6187039a636 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:29:42.514156 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-41dc97e9-5a8f-48de-b4b0-b6187039a636 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:29:42.528858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc97e9-5a8f-48de-b4b0-b6187039a636 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:29:42.535274 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ca9d-e0b7-40b0-bbd4-553fce80f1df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:29:42.536175 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ca9d-e0b7-40b0-bbd4-553fce80f1df service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:29:42.536461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ca9d-e0b7-40b0-bbd4-553fce80f1df service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:29:42.536727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ca9d-e0b7-40b0-bbd4-553fce80f1df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:29:42.539823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-aa95ca9d-e0b7-40b0-bbd4-553fce80f1df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:29:42.558423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95ca9d-e0b7-40b0-bbd4-553fce80f1df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:29:42.559881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95ca9d-e0b7-40b0-bbd4-553fce80f1df service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:29:42.571752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1233/2452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:29:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:29:51.332142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=54 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:29:52.566907 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=42 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:29:57.393444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b72d7b-7394-4e03-99d5-3eb34addc70b service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:29:57.403427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b72d7b-7394-4e03-99d5-3eb34addc70b service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:29:57.408343 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b9994-0ea7-4927-be7a-95a8e67c7549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:29:57.409017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b9994-0ea7-4927-be7a-95a8e67c7549 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:29:57.409292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b9994-0ea7-4927-be7a-95a8e67c7549 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:29:57.409559 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b9994-0ea7-4927-be7a-95a8e67c7549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:29:57.411941 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e19b9994-0ea7-4927-be7a-95a8e67c7549 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:29:57.423108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b9994-0ea7-4927-be7a-95a8e67c7549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:29:57.424608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b9994-0ea7-4927-be7a-95a8e67c7549 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:29:57.433797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1220/2453] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:29:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:30:07.430193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=6 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:30:44.384045 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ca9d-e0b7-40b0-bbd4-553fce80f1df service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:30:44.393686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ca9d-e0b7-40b0-bbd4-553fce80f1df service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:30:44.398005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba049dd-8d43-4654-aa9a-af79a305886a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:30:44.398674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba049dd-8d43-4654-aa9a-af79a305886a service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:30:44.398953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba049dd-8d43-4654-aa9a-af79a305886a service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:30:44.399223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba049dd-8d43-4654-aa9a-af79a305886a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:30:44.401217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-eba049dd-8d43-4654-aa9a-af79a305886a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:30:44.473127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba049dd-8d43-4654-aa9a-af79a305886a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:30:44.474652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba049dd-8d43-4654-aa9a-af79a305886a service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:30:44.494269 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1234/2454] 10.222.0.52 () {60 vars in 1396 bytes} [Wed Jun 29 10:30:44 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:30:54.491514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=36 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:31:40.949317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b9994-0ea7-4927-be7a-95a8e67c7549 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:31:40.959607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b9994-0ea7-4927-be7a-95a8e67c7549 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:31:40.964271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88843d30-59c2-436d-80e3-43fc71eb884a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:31:40.964917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88843d30-59c2-436d-80e3-43fc71eb884a service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:31:40.965187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88843d30-59c2-436d-80e3-43fc71eb884a service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:31:40.965455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88843d30-59c2-436d-80e3-43fc71eb884a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:31:40.967752 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-88843d30-59c2-436d-80e3-43fc71eb884a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:31:41.045189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88843d30-59c2-436d-80e3-43fc71eb884a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:31:41.046668 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88843d30-59c2-436d-80e3-43fc71eb884a service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:31:41.071319 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1221/2455] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:31:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:31:47.277346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba049dd-8d43-4654-aa9a-af79a305886a service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:31:47.278959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-eba049dd-8d43-4654-aa9a-af79a305886a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:31:47.295993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba049dd-8d43-4654-aa9a-af79a305886a service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:31:47.302149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10b826-88ad-4e7b-b599-7b2e9400586a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:31:47.302829 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10b826-88ad-4e7b-b599-7b2e9400586a service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:31:47.303116 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10b826-88ad-4e7b-b599-7b2e9400586a service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:31:47.303447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10b826-88ad-4e7b-b599-7b2e9400586a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:31:47.305340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7a10b826-88ad-4e7b-b599-7b2e9400586a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:31:47.321423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10b826-88ad-4e7b-b599-7b2e9400586a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:31:47.323169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10b826-88ad-4e7b-b599-7b2e9400586a service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:31:47.335260 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1235/2456] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:31:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:31:51.068242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=36 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:31:57.330511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:32:55.053705 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88843d30-59c2-436d-80e3-43fc71eb884a service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:32:55.077351 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88843d30-59c2-436d-80e3-43fc71eb884a service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:32:55.083218 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d296266-a7db-4d13-9fba-c5f80558af58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:32:55.083910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d296266-a7db-4d13-9fba-c5f80558af58 service placement] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:32:55.084217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d296266-a7db-4d13-9fba-c5f80558af58 service placement] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:32:55.084496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d296266-a7db-4d13-9fba-c5f80558af58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:32:55.093582 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d296266-a7db-4d13-9fba-c5f80558af58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:32:55.095313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d296266-a7db-4d13-9fba-c5f80558af58 service placement] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:32:55.105423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1222/2457] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:32:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:33:05.101139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=10 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:40.268302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8532391-1cdb-48fd-bcea-a9d61e8f8877 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:40.268959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8532391-1cdb-48fd-bcea-a9d61e8f8877 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:40.268959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8532391-1cdb-48fd-bcea-a9d61e8f8877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:40.340750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e8532391-1cdb-48fd-bcea-a9d61e8f8877 None None] MFA Rules not processed for user `43964a0e6b48487bb90c5356d312b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:40.341970 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e8532391-1cdb-48fd-bcea-a9d61e8f8877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:40.391065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1236/2458] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:33:40.418872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d296266-a7db-4d13-9fba-c5f80558af58 service placement] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:40.446931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d296266-a7db-4d13-9fba-c5f80558af58 service placement] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:40.450706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5790507-c078-48b2-99d1-c3375f1ae4ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:40.451354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5790507-c078-48b2-99d1-c3375f1ae4ca service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:40.451637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5790507-c078-48b2-99d1-c3375f1ae4ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:40.451914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5790507-c078-48b2-99d1-c3375f1ae4ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:40.453800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b5790507-c078-48b2-99d1-c3375f1ae4ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:40.585348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5790507-c078-48b2-99d1-c3375f1ae4ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:40.587330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5790507-c078-48b2-99d1-c3375f1ae4ca service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:40.612470 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1223/2459] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:33:40.731673 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8532391-1cdb-48fd-bcea-a9d61e8f8877 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:40.745097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8532391-1cdb-48fd-bcea-a9d61e8f8877 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:40.750900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99554b69-3a29-4bc8-9c5b-9fbf7c9adb95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:40.751770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99554b69-3a29-4bc8-9c5b-9fbf7c9adb95 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:40.751978 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99554b69-3a29-4bc8-9c5b-9fbf7c9adb95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:40.752144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99554b69-3a29-4bc8-9c5b-9fbf7c9adb95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:40.757011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-99554b69-3a29-4bc8-9c5b-9fbf7c9adb95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:40.788405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99554b69-3a29-4bc8-9c5b-9fbf7c9adb95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:40.789883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99554b69-3a29-4bc8-9c5b-9fbf7c9adb95 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:40.817024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1237/2460] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:33:41.547404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5790507-c078-48b2-99d1-c3375f1ae4ca service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:41.548856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b5790507-c078-48b2-99d1-c3375f1ae4ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:41.582251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5790507-c078-48b2-99d1-c3375f1ae4ca service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:41.587416 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa82dc-4e2b-4b8b-9b41-8f13bc3e2287 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:41.588078 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa82dc-4e2b-4b8b-9b41-8f13bc3e2287 service placement] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:41.588353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa82dc-4e2b-4b8b-9b41-8f13bc3e2287 service placement] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:41.589042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa82dc-4e2b-4b8b-9b41-8f13bc3e2287 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:41.591170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-23fa82dc-4e2b-4b8b-9b41-8f13bc3e2287 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:41.609281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa82dc-4e2b-4b8b-9b41-8f13bc3e2287 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:41.611135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa82dc-4e2b-4b8b-9b41-8f13bc3e2287 service placement] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:41.621894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1224/2461] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:33:42.780752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99554b69-3a29-4bc8-9c5b-9fbf7c9adb95 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:42.784088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-99554b69-3a29-4bc8-9c5b-9fbf7c9adb95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:42.856547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99554b69-3a29-4bc8-9c5b-9fbf7c9adb95 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:42.866217 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81612693-a010-450e-af17-5e1ccca975db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:42.866945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81612693-a010-450e-af17-5e1ccca975db service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:42.867225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81612693-a010-450e-af17-5e1ccca975db service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:42.867559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81612693-a010-450e-af17-5e1ccca975db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:42.869511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-81612693-a010-450e-af17-5e1ccca975db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:42.885289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81612693-a010-450e-af17-5e1ccca975db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:42.887000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81612693-a010-450e-af17-5e1ccca975db service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:42.901005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1238/2462] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:33:45.672113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa82dc-4e2b-4b8b-9b41-8f13bc3e2287 service placement] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:45.683966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa82dc-4e2b-4b8b-9b41-8f13bc3e2287 service placement] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:45.689982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f72d5-3ab1-491b-a0ba-0b43ba63e288 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:45.690676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f72d5-3ab1-491b-a0ba-0b43ba63e288 service placement] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:45.690959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f72d5-3ab1-491b-a0ba-0b43ba63e288 service placement] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:45.691234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f72d5-3ab1-491b-a0ba-0b43ba63e288 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:45.693631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-509f72d5-3ab1-491b-a0ba-0b43ba63e288 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:45.707082 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f72d5-3ab1-491b-a0ba-0b43ba63e288 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:45.708711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f72d5-3ab1-491b-a0ba-0b43ba63e288 service placement] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:45.721602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1225/2463] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:33:45.753141 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81612693-a010-450e-af17-5e1ccca975db service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:45.755773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-81612693-a010-450e-af17-5e1ccca975db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:45.773359 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81612693-a010-450e-af17-5e1ccca975db service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:45.778517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44b0fc-7d45-4eac-b94e-301e2e8faa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:45.779441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b0fc-7d45-4eac-b94e-301e2e8faa53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:45.779719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b0fc-7d45-4eac-b94e-301e2e8faa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:45.779992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b0fc-7d45-4eac-b94e-301e2e8faa53 admin admin] PATH_INFO: `/v3/users/7900ba750c5f4e9faa69baaed1fd95d7` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:45.781574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44b0fc-7d45-4eac-b94e-301e2e8faa53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7900ba750c5f4e9faa69baaed1fd95d7)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:45.783618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44b0fc-7d45-4eac-b94e-301e2e8faa53 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:45.834824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1c44b0fc-7d45-4eac-b94e-301e2e8faa53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7900ba750c5f4e9faa69baaed1fd95d7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:45.847033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1c44b0fc-7d45-4eac-b94e-301e2e8faa53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7900ba750c5f4e9faa69baaed1fd95d7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:45.847824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1c44b0fc-7d45-4eac-b94e-301e2e8faa53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7900ba750c5f4e9faa69baaed1fd95d7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:45.855113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1c44b0fc-7d45-4eac-b94e-301e2e8faa53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7900ba750c5f4e9faa69baaed1fd95d7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:45.855754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1c44b0fc-7d45-4eac-b94e-301e2e8faa53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7900ba750c5f4e9faa69baaed1fd95d7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:45.870029 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1239/2464] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:33:45 2022] DELETE /identity/v3/users/7900ba750c5f4e9faa69baaed1fd95d7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:46.049818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f72d5-3ab1-491b-a0ba-0b43ba63e288 service placement] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:46.063370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f72d5-3ab1-491b-a0ba-0b43ba63e288 service placement] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:46.146753 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:46.147658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:46.147938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:46.148267 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] PATH_INFO: `/v3/projects/f57102c066054370a153fb18dc4bc428` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:46.149505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f57102c066054370a153fb18dc4bc428)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:46.151477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:46.163686 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:46.206098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f57102c066054370a153fb18dc4bc428', 'request_id': 'req-17c39a52-204a-469e-be65-27fa537c94ad'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:46.215350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f57102c066054370a153fb18dc4bc428', 'request_id': 'req-17c39a52-204a-469e-be65-27fa537c94ad'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:46.222077 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f57102c066054370a153fb18dc4bc428', 'request_id': 'req-17c39a52-204a-469e-be65-27fa537c94ad'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:46.233328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] The token cache is being invalidate because project f57102c066054370a153fb18dc4bc428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:33:46.233606 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:46.234264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-17c39a52-204a-469e-be65-27fa537c94ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:46.236415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1226/2465] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:33:46 2022] DELETE /identity/v3/projects/f57102c066054370a153fb18dc4bc428 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:46.243414 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93964427-1b37-4bcf-a596-471d2fdba82d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:46.243770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93964427-1b37-4bcf-a596-471d2fdba82d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:46.244169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93964427-1b37-4bcf-a596-471d2fdba82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:46.249401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:46.317261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-93964427-1b37-4bcf-a596-471d2fdba82d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:46.318766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-93964427-1b37-4bcf-a596-471d2fdba82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:46.387120 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1240/2466] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:33:46.397706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8b343-3086-4466-8318-fcd18bb22a9e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:46.397915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8b343-3086-4466-8318-fcd18bb22a9e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:46.398169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8b343-3086-4466-8318-fcd18bb22a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:46.419303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2ea8b343-3086-4466-8318-fcd18bb22a9e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:46.420782 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2ea8b343-3086-4466-8318-fcd18bb22a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:46.429884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1227/2467] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:33:46.434394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93964427-1b37-4bcf-a596-471d2fdba82d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:46.444988 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93964427-1b37-4bcf-a596-471d2fdba82d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:46.450113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8a272-4f16-4bf1-b9b1-57edfd8845e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:46.450852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8a272-4f16-4bf1-b9b1-57edfd8845e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:46.451163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8a272-4f16-4bf1-b9b1-57edfd8845e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:46.451436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8a272-4f16-4bf1-b9b1-57edfd8845e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:46.452231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8a272-4f16-4bf1-b9b1-57edfd8845e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:33:46.452541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8a272-4f16-4bf1-b9b1-57edfd8845e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:46.454656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8a272-4f16-4bf1-b9b1-57edfd8845e7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:46.467949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1241/2468] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:33:46.473092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8b343-3086-4466-8318-fcd18bb22a9e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:46.483814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8b343-3086-4466-8318-fcd18bb22a9e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:46.490398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f805aaf-db09-4073-b100-ee4ece5405ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:46.491103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f805aaf-db09-4073-b100-ee4ece5405ef admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:46.491367 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f805aaf-db09-4073-b100-ee4ece5405ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:46.491641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f805aaf-db09-4073-b100-ee4ece5405ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:46.492344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f805aaf-db09-4073-b100-ee4ece5405ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:46.494266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f805aaf-db09-4073-b100-ee4ece5405ef admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:46.539326 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1228/2469] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:33:46 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:33:46.543173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8a272-4f16-4bf1-b9b1-57edfd8845e7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:46.558303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8a272-4f16-4bf1-b9b1-57edfd8845e7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:46.624707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1ffe2-4c91-45c5-8c67-4f79000e6aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:46.625486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1ffe2-4c91-45c5-8c67-4f79000e6aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:46.625813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1ffe2-4c91-45c5-8c67-4f79000e6aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:46.626158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1ffe2-4c91-45c5-8c67-4f79000e6aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:46.626906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1ffe2-4c91-45c5-8c67-4f79000e6aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:46.628826 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1ffe2-4c91-45c5-8c67-4f79000e6aa9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:46.656165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1242/2470] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:33:46 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:33:46.660194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f805aaf-db09-4073-b100-ee4ece5405ef admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:46.670874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f805aaf-db09-4073-b100-ee4ece5405ef admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:46.676283 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaedcc-ac69-4ad4-aaaa-f4f40cb731cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:46.676967 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaedcc-ac69-4ad4-aaaa-f4f40cb731cd admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:46.677270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaedcc-ac69-4ad4-aaaa-f4f40cb731cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:46.677542 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaedcc-ac69-4ad4-aaaa-f4f40cb731cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:46.678281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeaedcc-ac69-4ad4-aaaa-f4f40cb731cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:46.680012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeaedcc-ac69-4ad4-aaaa-f4f40cb731cd admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:46.694604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1229/2471] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 10:33:46.696268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1ffe2-4c91-45c5-8c67-4f79000e6aa9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:46.707698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1ffe2-4c91-45c5-8c67-4f79000e6aa9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:46.713107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf20767-26fe-43d9-8b96-49b8830dd3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:46.714127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf20767-26fe-43d9-8b96-49b8830dd3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:46.714527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf20767-26fe-43d9-8b96-49b8830dd3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:46.714903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf20767-26fe-43d9-8b96-49b8830dd3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:46.715980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf20767-26fe-43d9-8b96-49b8830dd3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:46.718175 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf20767-26fe-43d9-8b96-49b8830dd3a8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:46.732511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1243/2472] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:33:46.733875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaedcc-ac69-4ad4-aaaa-f4f40cb731cd admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:46.744774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaedcc-ac69-4ad4-aaaa-f4f40cb731cd admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:46.751087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b1880-5266-4ae3-9926-4819b2f2cc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:46.752037 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b1880-5266-4ae3-9926-4819b2f2cc42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:46.752320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b1880-5266-4ae3-9926-4819b2f2cc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:46.752581 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b1880-5266-4ae3-9926-4819b2f2cc42 admin admin] PATH_INFO: `/v3/projects/4cf74b22676841d0b0fb7feb2c5c9c87/users/c0029bc422014ff682d930ace15efd87/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:46.769332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b1880-5266-4ae3-9926-4819b2f2cc42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf74b22676841d0b0fb7feb2c5c9c87, user_id=c0029bc422014ff682d930ace15efd87, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:46.772767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b1880-5266-4ae3-9926-4819b2f2cc42 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:46.788133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1230/2473] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:33:46 2022] PUT /identity/v3/projects/4cf74b22676841d0b0fb7feb2c5c9c87/users/c0029bc422014ff682d930ace15efd87/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:46.799772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbabb17-00d3-46a3-8de8-923ec5c306b7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:46.800348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbabb17-00d3-46a3-8de8-923ec5c306b7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:46.800348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbabb17-00d3-46a3-8de8-923ec5c306b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:46.830089 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3cbabb17-00d3-46a3-8de8-923ec5c306b7 None None] MFA Rules not processed for user `c0029bc422014ff682d930ace15efd87`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:46.831403 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3cbabb17-00d3-46a3-8de8-923ec5c306b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:46.858920 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:46.900881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1244/2474] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:33:46.912483 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b1880-5266-4ae3-9926-4819b2f2cc42 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:46.914129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-159b1880-5266-4ae3-9926-4819b2f2cc42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:46.993248 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b1880-5266-4ae3-9926-4819b2f2cc42 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:46.999707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770ce88-e9cd-4405-9faf-cf307dcb59c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:47.000372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770ce88-e9cd-4405-9faf-cf307dcb59c7 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:47.000640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770ce88-e9cd-4405-9faf-cf307dcb59c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:47.000919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770ce88-e9cd-4405-9faf-cf307dcb59c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:47.009053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770ce88-e9cd-4405-9faf-cf307dcb59c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:47.010658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770ce88-e9cd-4405-9faf-cf307dcb59c7 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:47.020081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1231/2475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:33:49.808515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbabb17-00d3-46a3-8de8-923ec5c306b7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:49.877052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbabb17-00d3-46a3-8de8-923ec5c306b7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:49.881703 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:49.882611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:49.882892 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:49.883170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] PATH_INFO: `/v3/users/c0029bc422014ff682d930ace15efd87` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:49.884411 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0029bc422014ff682d930ace15efd87)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:49.886350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:49.954209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0029bc422014ff682d930ace15efd87'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:49.968438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0029bc422014ff682d930ace15efd87'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:49.969456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0029bc422014ff682d930ace15efd87'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:49.978234 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0029bc422014ff682d930ace15efd87'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:49.979340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0029bc422014ff682d930ace15efd87'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:49.995976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1245/2476] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:33:49 2022] DELETE /identity/v3/users/c0029bc422014ff682d930ace15efd87 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:50.001370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7009f-5994-4181-8ef5-993fb7b319d7 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:50.002309 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7009f-5994-4181-8ef5-993fb7b319d7 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:50.002602 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7009f-5994-4181-8ef5-993fb7b319d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:50.021505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-91f7009f-5994-4181-8ef5-993fb7b319d7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:50.022747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-91f7009f-5994-4181-8ef5-993fb7b319d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:50.037538 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:50.129055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1232/2477] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:33:50.138764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:50.140224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:50.205309 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:50.213823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f4562-c825-483c-a2f5-53a79f189813 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:50.220567 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa1bd5-e3d1-45b6-88d9-d93d18ada275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:50.221795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa1bd5-e3d1-45b6-88d9-d93d18ada275 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:50.221795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa1bd5-e3d1-45b6-88d9-d93d18ada275 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:50.222511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa1bd5-e3d1-45b6-88d9-d93d18ada275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:50.234686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa1bd5-e3d1-45b6-88d9-d93d18ada275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:50.237435 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa1bd5-e3d1-45b6-88d9-d93d18ada275 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:50.250282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1246/2478] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:33:50.616669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7009f-5994-4181-8ef5-993fb7b319d7 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:50.619288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-91f7009f-5994-4181-8ef5-993fb7b319d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:50.634110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7009f-5994-4181-8ef5-993fb7b319d7 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:50.639195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:50.640104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:50.640434 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:50.640666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] PATH_INFO: `/v3/projects/4cf74b22676841d0b0fb7feb2c5c9c87` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:50.641989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cf74b22676841d0b0fb7feb2c5c9c87)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:50.644024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:50.707901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cf74b22676841d0b0fb7feb2c5c9c87', 'request_id': 'req-e903c424-bec9-4d31-8255-aaf842ea6859'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:50.719042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cf74b22676841d0b0fb7feb2c5c9c87', 'request_id': 'req-e903c424-bec9-4d31-8255-aaf842ea6859'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:50.726514 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cf74b22676841d0b0fb7feb2c5c9c87', 'request_id': 'req-e903c424-bec9-4d31-8255-aaf842ea6859'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:50.740802 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] The token cache is being invalidate because project 4cf74b22676841d0b0fb7feb2c5c9c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:33:50.741298 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:50.742306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e903c424-bec9-4d31-8255-aaf842ea6859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:50.744479 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1233/2479] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:33:50 2022] DELETE /identity/v3/projects/4cf74b22676841d0b0fb7feb2c5c9c87 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:50.749835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007aed-6aa9-46f9-a959-aa8d4b687d47 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:50.750224 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007aed-6aa9-46f9-a959-aa8d4b687d47 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:50.750540 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007aed-6aa9-46f9-a959-aa8d4b687d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:50.773690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d5007aed-6aa9-46f9-a959-aa8d4b687d47 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:50.775708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d5007aed-6aa9-46f9-a959-aa8d4b687d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:50.880800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1247/2480] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:33:50.889758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69d2d7-cca6-44e1-a24d-8e7a5ec9b879 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:50.890054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69d2d7-cca6-44e1-a24d-8e7a5ec9b879 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:50.890469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69d2d7-cca6-44e1-a24d-8e7a5ec9b879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:50.913501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-8e69d2d7-cca6-44e1-a24d-8e7a5ec9b879 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:50.915244 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8e69d2d7-cca6-44e1-a24d-8e7a5ec9b879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:50.925402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1234/2481] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:33:50.930428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5007aed-6aa9-46f9-a959-aa8d4b687d47 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:50.941650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5007aed-6aa9-46f9-a959-aa8d4b687d47 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:50.947555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8043b-1d03-4d1f-ba01-1c208e21a9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:50.948367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8043b-1d03-4d1f-ba01-1c208e21a9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:50.948578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8043b-1d03-4d1f-ba01-1c208e21a9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:50.948870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8043b-1d03-4d1f-ba01-1c208e21a9b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:50.949635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8043b-1d03-4d1f-ba01-1c208e21a9b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:33:50.949962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8043b-1d03-4d1f-ba01-1c208e21a9b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:50.951709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8043b-1d03-4d1f-ba01-1c208e21a9b9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:50.968157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1248/2482] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 10:33:50.972747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69d2d7-cca6-44e1-a24d-8e7a5ec9b879 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:50.982664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69d2d7-cca6-44e1-a24d-8e7a5ec9b879 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:50.987876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8063b3-0d79-4731-a933-6065faba44a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:50.988336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8063b3-0d79-4731-a933-6065faba44a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:50.988821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8063b3-0d79-4731-a933-6065faba44a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:50.988821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8063b3-0d79-4731-a933-6065faba44a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:50.989521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8063b3-0d79-4731-a933-6065faba44a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:50.991522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8063b3-0d79-4731-a933-6065faba44a0 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:51.024563 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1235/2483] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:33:50 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:33:51.030122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8043b-1d03-4d1f-ba01-1c208e21a9b9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:51.042054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8043b-1d03-4d1f-ba01-1c208e21a9b9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:51.112939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ed1a3-9fae-4798-a622-2d436f9bda5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:51.113702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ed1a3-9fae-4798-a622-2d436f9bda5f admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.113968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ed1a3-9fae-4798-a622-2d436f9bda5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.114263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ed1a3-9fae-4798-a622-2d436f9bda5f admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.115127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ed1a3-9fae-4798-a622-2d436f9bda5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:51.117436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ed1a3-9fae-4798-a622-2d436f9bda5f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:51.141486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1249/2484] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:33:51 2022] POST /identity/v3/users => generated 285 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:33:51.145995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8063b3-0d79-4731-a933-6065faba44a0 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:51.157642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8063b3-0d79-4731-a933-6065faba44a0 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:51.163723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c658be4-7d39-41d2-af29-ba4c96964064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:51.164467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c658be4-7d39-41d2-af29-ba4c96964064 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.164738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c658be4-7d39-41d2-af29-ba4c96964064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.165014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c658be4-7d39-41d2-af29-ba4c96964064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.166097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c658be4-7d39-41d2-af29-ba4c96964064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:51.167758 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c658be4-7d39-41d2-af29-ba4c96964064 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:51.180546 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1236/2485] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 10:33:51.185689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ed1a3-9fae-4798-a622-2d436f9bda5f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:51.200828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ed1a3-9fae-4798-a622-2d436f9bda5f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:51.207815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f8c7a-cb19-4595-9db3-baa3a18c91e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:51.208496 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f8c7a-cb19-4595-9db3-baa3a18c91e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.208772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f8c7a-cb19-4595-9db3-baa3a18c91e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.209043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f8c7a-cb19-4595-9db3-baa3a18c91e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.209773 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f8c7a-cb19-4595-9db3-baa3a18c91e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:51.211628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f8c7a-cb19-4595-9db3-baa3a18c91e6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:51.230689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1250/2486] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:33:51.233062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c658be4-7d39-41d2-af29-ba4c96964064 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:51.245331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c658be4-7d39-41d2-af29-ba4c96964064 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:51.251766 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33efddd-4d99-468a-b028-1fb08d17bf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:51.252699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33efddd-4d99-468a-b028-1fb08d17bf3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.252973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33efddd-4d99-468a-b028-1fb08d17bf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.253264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33efddd-4d99-468a-b028-1fb08d17bf3f admin admin] PATH_INFO: `/v3/projects/052067ae42e348c3abce338f50cc9a2b/users/bfe12071b8af4a93bc60542b3379e124/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.270496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33efddd-4d99-468a-b028-1fb08d17bf3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=052067ae42e348c3abce338f50cc9a2b, user_id=bfe12071b8af4a93bc60542b3379e124, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:51.273215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33efddd-4d99-468a-b028-1fb08d17bf3f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:51.290084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1237/2487] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:33:51 2022] PUT /identity/v3/projects/052067ae42e348c3abce338f50cc9a2b/users/bfe12071b8af4a93bc60542b3379e124/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:33:51.302912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdb4a-4190-46d8-8ea1-08e030534f24 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.302912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdb4a-4190-46d8-8ea1-08e030534f24 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.302912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdb4a-4190-46d8-8ea1-08e030534f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.336869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-932fdb4a-4190-46d8-8ea1-08e030534f24 None None] MFA Rules not processed for user `bfe12071b8af4a93bc60542b3379e124`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:51.338098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-932fdb4a-4190-46d8-8ea1-08e030534f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:51.394494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:51.417628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1251/2488] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:33:51.422759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33efddd-4d99-468a-b028-1fb08d17bf3f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:51.465387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:51.512745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33efddd-4d99-468a-b028-1fb08d17bf3f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:51.520233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba427a01-1450-4c38-a708-2815232397c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:51.520922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba427a01-1450-4c38-a708-2815232397c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.521204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba427a01-1450-4c38-a708-2815232397c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.521472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba427a01-1450-4c38-a708-2815232397c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.522173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba427a01-1450-4c38-a708-2815232397c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:51.524224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba427a01-1450-4c38-a708-2815232397c9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:51.559982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1238/2489] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:33:51 2022] POST /identity/v3/projects => generated 369 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:33:51.565769 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fdb4a-4190-46d8-8ea1-08e030534f24 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:51.577530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fdb4a-4190-46d8-8ea1-08e030534f24 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:51.641671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056aa030-d09e-4297-8506-23faef67e359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:51.642594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056aa030-d09e-4297-8506-23faef67e359 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.643347 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056aa030-d09e-4297-8506-23faef67e359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.643709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056aa030-d09e-4297-8506-23faef67e359 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.644622 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056aa030-d09e-4297-8506-23faef67e359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:51.647051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056aa030-d09e-4297-8506-23faef67e359 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:51.669542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1252/2490] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:33:51 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:33:51.673997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba427a01-1450-4c38-a708-2815232397c9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:51.685761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba427a01-1450-4c38-a708-2815232397c9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:51.691472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc02abb-bd89-4ff2-81b6-1bf323394359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:51.692150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc02abb-bd89-4ff2-81b6-1bf323394359 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.692426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc02abb-bd89-4ff2-81b6-1bf323394359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.692697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc02abb-bd89-4ff2-81b6-1bf323394359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.693425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc02abb-bd89-4ff2-81b6-1bf323394359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:51.695122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc02abb-bd89-4ff2-81b6-1bf323394359 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:51.706999 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1239/2491] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:33:51.711413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056aa030-d09e-4297-8506-23faef67e359 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:51.723341 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056aa030-d09e-4297-8506-23faef67e359 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:51.728687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb39130-7726-4a2b-a3c2-c464f72b08ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:51.729603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39130-7726-4a2b-a3c2-c464f72b08ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.729930 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39130-7726-4a2b-a3c2-c464f72b08ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.730152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39130-7726-4a2b-a3c2-c464f72b08ae admin admin] PATH_INFO: `/v3/projects/3e93dbb213e74cd7b527d0f929599182/users/d836575ad1e74687a9ec41492af28372/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.748880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb39130-7726-4a2b-a3c2-c464f72b08ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e93dbb213e74cd7b527d0f929599182, user_id=d836575ad1e74687a9ec41492af28372, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:51.752293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb39130-7726-4a2b-a3c2-c464f72b08ae admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:51.774180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1253/2492] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:33:51 2022] PUT /identity/v3/projects/3e93dbb213e74cd7b527d0f929599182/users/d836575ad1e74687a9ec41492af28372/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:51.778639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc02abb-bd89-4ff2-81b6-1bf323394359 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:51.791065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc02abb-bd89-4ff2-81b6-1bf323394359 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:51.857986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826fdee-438c-49a3-9635-0514a597f8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:51.858712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826fdee-438c-49a3-9635-0514a597f8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.858972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826fdee-438c-49a3-9635-0514a597f8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.859306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826fdee-438c-49a3-9635-0514a597f8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.860185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826fdee-438c-49a3-9635-0514a597f8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:51.862047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826fdee-438c-49a3-9635-0514a597f8e7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:51.873932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1240/2493] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:33:51.880660 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb39130-7726-4a2b-a3c2-c464f72b08ae admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:51.897254 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb39130-7726-4a2b-a3c2-c464f72b08ae admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:51.902396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112fcb94-7bb3-43ee-9f6a-cee790a9cd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:51.903356 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fcb94-7bb3-43ee-9f6a-cee790a9cd65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.903635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fcb94-7bb3-43ee-9f6a-cee790a9cd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.903903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fcb94-7bb3-43ee-9f6a-cee790a9cd65 admin admin] PATH_INFO: `/v3/projects/3e93dbb213e74cd7b527d0f929599182/users/d836575ad1e74687a9ec41492af28372/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.905888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112fcb94-7bb3-43ee-9f6a-cee790a9cd65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e93dbb213e74cd7b527d0f929599182, user_id=d836575ad1e74687a9ec41492af28372, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:51.908691 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112fcb94-7bb3-43ee-9f6a-cee790a9cd65 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:51.922028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1254/2494] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:33:51 2022] PUT /identity/v3/projects/3e93dbb213e74cd7b527d0f929599182/users/d836575ad1e74687a9ec41492af28372/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:51.933452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db94be-77f5-4ab9-9749-814d34d4f5e8 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:51.933721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db94be-77f5-4ab9-9749-814d34d4f5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:51.934000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db94be-77f5-4ab9-9749-814d34d4f5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:51.983747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-c3db94be-77f5-4ab9-9749-814d34d4f5e8 None None] MFA Rules not processed for user `d836575ad1e74687a9ec41492af28372`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:51.985513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c3db94be-77f5-4ab9-9749-814d34d4f5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:52.071426 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1241/2495] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:51 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:33:52.091572 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112fcb94-7bb3-43ee-9f6a-cee790a9cd65 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:52.093896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-112fcb94-7bb3-43ee-9f6a-cee790a9cd65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:52.137025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db94be-77f5-4ab9-9749-814d34d4f5e8 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:52.139393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c3db94be-77f5-4ab9-9749-814d34d4f5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:52.179226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112fcb94-7bb3-43ee-9f6a-cee790a9cd65 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:52.184502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ae457-286b-4cd6-b344-43d3f2578875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:52.185508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae457-286b-4cd6-b344-43d3f2578875 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:52.185508 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae457-286b-4cd6-b344-43d3f2578875 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:52.186824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae457-286b-4cd6-b344-43d3f2578875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:52.189278 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:52.198981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ae457-286b-4cd6-b344-43d3f2578875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:52.200576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ae457-286b-4cd6-b344-43d3f2578875 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:52.213812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1255/2496] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:52 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:33:52.247097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db94be-77f5-4ab9-9749-814d34d4f5e8 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:52.254770 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5564b63-77f9-427a-9550-5e073acd27c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:52.255487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5564b63-77f9-427a-9550-5e073acd27c1 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:52.255787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5564b63-77f9-427a-9550-5e073acd27c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:52.256083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5564b63-77f9-427a-9550-5e073acd27c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:52.257827 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d5564b63-77f9-427a-9550-5e073acd27c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:52.262632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ae457-286b-4cd6-b344-43d3f2578875 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:52.264105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-458ae457-286b-4cd6-b344-43d3f2578875 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:52.275478 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5564b63-77f9-427a-9550-5e073acd27c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:52.278312 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5564b63-77f9-427a-9550-5e073acd27c1 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:52.281753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ae457-286b-4cd6-b344-43d3f2578875 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:52.285799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff742e99-d2c1-407c-aced-5fccbd62a167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:52.286536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff742e99-d2c1-407c-aced-5fccbd62a167 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:52.286848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff742e99-d2c1-407c-aced-5fccbd62a167 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:52.287118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff742e99-d2c1-407c-aced-5fccbd62a167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:52.290243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1242/2497] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:33:52.340448 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:52.363572 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff742e99-d2c1-407c-aced-5fccbd62a167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:52.365068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff742e99-d2c1-407c-aced-5fccbd62a167 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:52.376666 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1256/2498] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:33:55.944323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5564b63-77f9-427a-9550-5e073acd27c1 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:56.014882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5564b63-77f9-427a-9550-5e073acd27c1 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:56.019990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:56.020881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:56.021160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:56.021439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] PATH_INFO: `/v3/users/bfe12071b8af4a93bc60542b3379e124` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:56.022718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfe12071b8af4a93bc60542b3379e124)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:56.024781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:56.077373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfe12071b8af4a93bc60542b3379e124'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.091407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfe12071b8af4a93bc60542b3379e124'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.092163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfe12071b8af4a93bc60542b3379e124'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.092759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfe12071b8af4a93bc60542b3379e124'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.101275 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfe12071b8af4a93bc60542b3379e124'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.115427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1243/2499] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:33:55 2022] DELETE /identity/v3/users/bfe12071b8af4a93bc60542b3379e124 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:56.128620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda61de-3839-412f-833f-dd3c9684801d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:56.128620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda61de-3839-412f-833f-dd3c9684801d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:56.128620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda61de-3839-412f-833f-dd3c9684801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:56.139925 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4fda61de-3839-412f-833f-dd3c9684801d None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:56.141316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4fda61de-3839-412f-833f-dd3c9684801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:56.203612 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1257/2500] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:33:56.214611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:56.216270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:56.258708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:56.289626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaf9fe-3827-47d5-91e7-aa58b86a2eeb admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:56.293794 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb48a15-1856-4c34-ac39-297a7ffdd9b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:56.294468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb48a15-1856-4c34-ac39-297a7ffdd9b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:56.294924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb48a15-1856-4c34-ac39-297a7ffdd9b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:56.295199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb48a15-1856-4c34-ac39-297a7ffdd9b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:56.311505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb48a15-1856-4c34-ac39-297a7ffdd9b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:56.313120 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb48a15-1856-4c34-ac39-297a7ffdd9b1 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:56.336595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1244/2501] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:33:56.644397 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda61de-3839-412f-833f-dd3c9684801d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:56.646059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4fda61de-3839-412f-833f-dd3c9684801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:56.661553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda61de-3839-412f-833f-dd3c9684801d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:56.669799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:56.670720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:56.671106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:56.671547 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] PATH_INFO: `/v3/projects/052067ae42e348c3abce338f50cc9a2b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:56.672626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] RBAC: Authorizing `identity:delete_project(project_id=052067ae42e348c3abce338f50cc9a2b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:56.680405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:56.743980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '052067ae42e348c3abce338f50cc9a2b', 'request_id': 'req-1a39986e-2c09-411e-acb6-48da5dfcd89c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.753116 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '052067ae42e348c3abce338f50cc9a2b', 'request_id': 'req-1a39986e-2c09-411e-acb6-48da5dfcd89c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.759586 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '052067ae42e348c3abce338f50cc9a2b', 'request_id': 'req-1a39986e-2c09-411e-acb6-48da5dfcd89c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.786714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] The token cache is being invalidate because project 052067ae42e348c3abce338f50cc9a2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:33:56.787007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.787796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.789518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1258/2502] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:33:56 2022] DELETE /identity/v3/projects/052067ae42e348c3abce338f50cc9a2b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:56.794680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb48a15-1856-4c34-ac39-297a7ffdd9b1 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:56.805847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb48a15-1856-4c34-ac39-297a7ffdd9b1 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:56.870348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa94d80-743f-4b35-b8aa-f94b70f1f956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:56.871170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa94d80-743f-4b35-b8aa-f94b70f1f956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:56.871446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa94d80-743f-4b35-b8aa-f94b70f1f956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:56.871717 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa94d80-743f-4b35-b8aa-f94b70f1f956 admin admin] PATH_INFO: `/v3/users/d836575ad1e74687a9ec41492af28372` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:56.872959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa94d80-743f-4b35-b8aa-f94b70f1f956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d836575ad1e74687a9ec41492af28372)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:56.874982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa94d80-743f-4b35-b8aa-f94b70f1f956 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:56.932451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-baa94d80-743f-4b35-b8aa-f94b70f1f956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd836575ad1e74687a9ec41492af28372'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.946595 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-baa94d80-743f-4b35-b8aa-f94b70f1f956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd836575ad1e74687a9ec41492af28372'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.947443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-baa94d80-743f-4b35-b8aa-f94b70f1f956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd836575ad1e74687a9ec41492af28372'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.948123 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-baa94d80-743f-4b35-b8aa-f94b70f1f956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd836575ad1e74687a9ec41492af28372'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.956776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-baa94d80-743f-4b35-b8aa-f94b70f1f956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd836575ad1e74687a9ec41492af28372'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:56.981176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1245/2503] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:33:56 2022] DELETE /identity/v3/users/d836575ad1e74687a9ec41492af28372 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:57.291650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:57.293181 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:57.312335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39986e-2c09-411e-acb6-48da5dfcd89c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:57.403969 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:57.404900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:57.405175 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:57.405446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] PATH_INFO: `/v3/projects/3e93dbb213e74cd7b527d0f929599182` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:57.406727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e93dbb213e74cd7b527d0f929599182)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:57.408811 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:57.411997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:57.468579 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e93dbb213e74cd7b527d0f929599182', 'request_id': 'req-b8d6ca61-5501-40bb-987a-951a5bafb34e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:57.478193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e93dbb213e74cd7b527d0f929599182', 'request_id': 'req-b8d6ca61-5501-40bb-987a-951a5bafb34e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:57.486239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e93dbb213e74cd7b527d0f929599182', 'request_id': 'req-b8d6ca61-5501-40bb-987a-951a5bafb34e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:57.501178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] The token cache is being invalidate because project 3e93dbb213e74cd7b527d0f929599182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:33:57.501178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:57.501973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b8d6ca61-5501-40bb-987a-951a5bafb34e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:33:57.504731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1259/2504] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:33:57 2022] DELETE /identity/v3/projects/3e93dbb213e74cd7b527d0f929599182 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:57.509890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0184ed-0010-4b02-a654-72fc746a58d1 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:57.510164 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0184ed-0010-4b02-a654-72fc746a58d1 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:57.510465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0184ed-0010-4b02-a654-72fc746a58d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:57.531157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-ea0184ed-0010-4b02-a654-72fc746a58d1 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:57.532551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ea0184ed-0010-4b02-a654-72fc746a58d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:57.593080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:57.611702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1246/2505] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:33:57.624056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81948977-170d-43b7-98eb-c8f2b2108273 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:57.624528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81948977-170d-43b7-98eb-c8f2b2108273 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:57.624995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81948977-170d-43b7-98eb-c8f2b2108273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:57.651130 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-81948977-170d-43b7-98eb-c8f2b2108273 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:57.653514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-81948977-170d-43b7-98eb-c8f2b2108273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:57.665878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1260/2506] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:33:57.670536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0184ed-0010-4b02-a654-72fc746a58d1 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:57.681310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0184ed-0010-4b02-a654-72fc746a58d1 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:57.686774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075fc0f-b841-4e66-9b7d-d1e233df0201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:57.687455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075fc0f-b841-4e66-9b7d-d1e233df0201 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:57.687735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075fc0f-b841-4e66-9b7d-d1e233df0201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:57.688006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075fc0f-b841-4e66-9b7d-d1e233df0201 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:57.688740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075fc0f-b841-4e66-9b7d-d1e233df0201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:33:57.689049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075fc0f-b841-4e66-9b7d-d1e233df0201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:57.690742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075fc0f-b841-4e66-9b7d-d1e233df0201 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:57.704349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1247/2507] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:33:57.709094 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81948977-170d-43b7-98eb-c8f2b2108273 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:57.719578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81948977-170d-43b7-98eb-c8f2b2108273 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:57.726102 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609e4e0-a231-49b6-ae5b-d141f793b875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:57.726824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609e4e0-a231-49b6-ae5b-d141f793b875 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:57.727106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609e4e0-a231-49b6-ae5b-d141f793b875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:57.727847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609e4e0-a231-49b6-ae5b-d141f793b875 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:57.728086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609e4e0-a231-49b6-ae5b-d141f793b875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:57.730015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609e4e0-a231-49b6-ae5b-d141f793b875 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:57.762474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1261/2508] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:33:57 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:33:57.767506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075fc0f-b841-4e66-9b7d-d1e233df0201 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:57.780009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075fc0f-b841-4e66-9b7d-d1e233df0201 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:57.847669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750be7bb-d1be-442c-aeb9-0865d57c014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:57.848185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750be7bb-d1be-442c-aeb9-0865d57c014e admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:57.848934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750be7bb-d1be-442c-aeb9-0865d57c014e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:57.848934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750be7bb-d1be-442c-aeb9-0865d57c014e admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:57.849411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750be7bb-d1be-442c-aeb9-0865d57c014e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:57.851330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750be7bb-d1be-442c-aeb9-0865d57c014e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:57.873382 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1248/2509] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:33:57 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:33:57.877737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609e4e0-a231-49b6-ae5b-d141f793b875 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:57.889408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609e4e0-a231-49b6-ae5b-d141f793b875 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:57.894438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4f73e-9ba3-478d-8142-cf9ef5ebff47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:57.895183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4f73e-9ba3-478d-8142-cf9ef5ebff47 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:57.895467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4f73e-9ba3-478d-8142-cf9ef5ebff47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:57.895744 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4f73e-9ba3-478d-8142-cf9ef5ebff47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:57.896512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4f73e-9ba3-478d-8142-cf9ef5ebff47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:57.898198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4f73e-9ba3-478d-8142-cf9ef5ebff47 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:57.911450 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1262/2510] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:33:57.915942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750be7bb-d1be-442c-aeb9-0865d57c014e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:57.928667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750be7bb-d1be-442c-aeb9-0865d57c014e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:57.934532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26845c9f-914e-4eeb-9754-a1f83f7fc66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:57.935427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26845c9f-914e-4eeb-9754-a1f83f7fc66c admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:57.935708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26845c9f-914e-4eeb-9754-a1f83f7fc66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:57.935980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26845c9f-914e-4eeb-9754-a1f83f7fc66c admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:57.936712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26845c9f-914e-4eeb-9754-a1f83f7fc66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:57.938324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26845c9f-914e-4eeb-9754-a1f83f7fc66c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:57.949776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1249/2511] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:33:57.954464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4f73e-9ba3-478d-8142-cf9ef5ebff47 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:57.966353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4f73e-9ba3-478d-8142-cf9ef5ebff47 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:57.971382 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397c523-891c-4032-9b7a-2b83917500c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:57.972903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397c523-891c-4032-9b7a-2b83917500c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:57.973268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397c523-891c-4032-9b7a-2b83917500c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:57.973563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397c523-891c-4032-9b7a-2b83917500c0 admin admin] PATH_INFO: `/v3/projects/0622aec4ed3d4381a6911baf5bce8e8d/users/eea05e0acc09483f8b5a97ca5ce8d5ea/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:57.993854 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397c523-891c-4032-9b7a-2b83917500c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0622aec4ed3d4381a6911baf5bce8e8d, user_id=eea05e0acc09483f8b5a97ca5ce8d5ea, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:57.996900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397c523-891c-4032-9b7a-2b83917500c0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:58.015243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1263/2512] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:33:57 2022] PUT /identity/v3/projects/0622aec4ed3d4381a6911baf5bce8e8d/users/eea05e0acc09483f8b5a97ca5ce8d5ea/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:58.083936 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544be3f7-35ad-47df-bcac-b017202d5e16 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:58.084251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544be3f7-35ad-47df-bcac-b017202d5e16 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:58.084638 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544be3f7-35ad-47df-bcac-b017202d5e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:58.120949 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-544be3f7-35ad-47df-bcac-b017202d5e16 None None] MFA Rules not processed for user `eea05e0acc09483f8b5a97ca5ce8d5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:58.123015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-544be3f7-35ad-47df-bcac-b017202d5e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:58.207265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1250/2513] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:33:58.212415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397c523-891c-4032-9b7a-2b83917500c0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:58.322331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397c523-891c-4032-9b7a-2b83917500c0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:58.328038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7146f-2fa0-4171-be36-4dfdf9b2f510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:58.328789 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7146f-2fa0-4171-be36-4dfdf9b2f510 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:58.329142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7146f-2fa0-4171-be36-4dfdf9b2f510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:58.329483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7146f-2fa0-4171-be36-4dfdf9b2f510 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:58.330248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7146f-2fa0-4171-be36-4dfdf9b2f510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:58.332684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7146f-2fa0-4171-be36-4dfdf9b2f510 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:58.373876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1264/2514] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:33:58 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:33:58.378946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544be3f7-35ad-47df-bcac-b017202d5e16 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:58.390346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544be3f7-35ad-47df-bcac-b017202d5e16 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:58.445572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:58.460080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba294422-5170-4c8b-96ed-fe3c32eef989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:58.460844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba294422-5170-4c8b-96ed-fe3c32eef989 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:58.461181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba294422-5170-4c8b-96ed-fe3c32eef989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:58.461523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba294422-5170-4c8b-96ed-fe3c32eef989 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:58.462450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba294422-5170-4c8b-96ed-fe3c32eef989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:58.473112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba294422-5170-4c8b-96ed-fe3c32eef989 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:58.487552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1251/2515] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:33:58 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:33:58.492329 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7146f-2fa0-4171-be36-4dfdf9b2f510 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:58.503938 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7146f-2fa0-4171-be36-4dfdf9b2f510 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:58.509042 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe127f2d-af1e-43da-a430-11067d907731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:58.509965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe127f2d-af1e-43da-a430-11067d907731 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:58.510245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe127f2d-af1e-43da-a430-11067d907731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:58.510517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe127f2d-af1e-43da-a430-11067d907731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:58.511285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe127f2d-af1e-43da-a430-11067d907731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:58.512959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe127f2d-af1e-43da-a430-11067d907731 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:58.525499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1265/2516] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:33:58.530387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba294422-5170-4c8b-96ed-fe3c32eef989 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:58.545142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba294422-5170-4c8b-96ed-fe3c32eef989 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:58.552621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405c797-c77f-4796-8695-cf235b77cbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:58.553530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405c797-c77f-4796-8695-cf235b77cbd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:58.553813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405c797-c77f-4796-8695-cf235b77cbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:58.554088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405c797-c77f-4796-8695-cf235b77cbd1 admin admin] PATH_INFO: `/v3/projects/c4b04072d2e34cc88754dad66731f45f/users/0a71051dc5bf4c449e66065ae2aa370a/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:58.573072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405c797-c77f-4796-8695-cf235b77cbd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b04072d2e34cc88754dad66731f45f, user_id=0a71051dc5bf4c449e66065ae2aa370a, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:58.576438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405c797-c77f-4796-8695-cf235b77cbd1 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:58.591740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1252/2517] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:33:58 2022] PUT /identity/v3/projects/c4b04072d2e34cc88754dad66731f45f/users/0a71051dc5bf4c449e66065ae2aa370a/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:58.596071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe127f2d-af1e-43da-a430-11067d907731 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:58.602681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:33:58.609416 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe127f2d-af1e-43da-a430-11067d907731 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:58.674592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab5d9f-b653-4dd9-b19d-caa5fe43e314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:58.675354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab5d9f-b653-4dd9-b19d-caa5fe43e314 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:58.675740 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab5d9f-b653-4dd9-b19d-caa5fe43e314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:58.676002 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab5d9f-b653-4dd9-b19d-caa5fe43e314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:58.676764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab5d9f-b653-4dd9-b19d-caa5fe43e314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:58.678380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab5d9f-b653-4dd9-b19d-caa5fe43e314 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:58.692609 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1266/2518] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:33:58.697742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405c797-c77f-4796-8695-cf235b77cbd1 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:58.709453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405c797-c77f-4796-8695-cf235b77cbd1 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:58.714999 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97094dbd-1a7b-48bf-be48-11720a6674cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:58.715886 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97094dbd-1a7b-48bf-be48-11720a6674cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:58.716164 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97094dbd-1a7b-48bf-be48-11720a6674cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:58.716437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97094dbd-1a7b-48bf-be48-11720a6674cb admin admin] PATH_INFO: `/v3/projects/c4b04072d2e34cc88754dad66731f45f/users/0a71051dc5bf4c449e66065ae2aa370a/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:58.718853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97094dbd-1a7b-48bf-be48-11720a6674cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b04072d2e34cc88754dad66731f45f, user_id=0a71051dc5bf4c449e66065ae2aa370a, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:58.721513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97094dbd-1a7b-48bf-be48-11720a6674cb admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:58.733014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1253/2519] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:33:58 2022] PUT /identity/v3/projects/c4b04072d2e34cc88754dad66731f45f/users/0a71051dc5bf4c449e66065ae2aa370a/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:33:58.743972 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52815b7-2449-4a02-a7e5-227e89528260 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:58.744207 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52815b7-2449-4a02-a7e5-227e89528260 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:58.744512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52815b7-2449-4a02-a7e5-227e89528260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:58.775007 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d52815b7-2449-4a02-a7e5-227e89528260 None None] MFA Rules not processed for user `0a71051dc5bf4c449e66065ae2aa370a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:33:58.776287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d52815b7-2449-4a02-a7e5-227e89528260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:58.855423 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1267/2520] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:33:58.866828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97094dbd-1a7b-48bf-be48-11720a6674cb admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:33:58.868813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-97094dbd-1a7b-48bf-be48-11720a6674cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:33:58.934898 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97094dbd-1a7b-48bf-be48-11720a6674cb admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:33:58.939430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61b4f5-288c-4682-96c9-89b942c08439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:33:58.940092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61b4f5-288c-4682-96c9-89b942c08439 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:33:58.940374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61b4f5-288c-4682-96c9-89b942c08439 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:33:58.940634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61b4f5-288c-4682-96c9-89b942c08439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:33:59.004324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61b4f5-288c-4682-96c9-89b942c08439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:33:59.005623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61b4f5-288c-4682-96c9-89b942c08439 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:33:59.014465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1254/2521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:33:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:03.434736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52815b7-2449-4a02-a7e5-227e89528260 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:03.436721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d52815b7-2449-4a02-a7e5-227e89528260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:03.450973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52815b7-2449-4a02-a7e5-227e89528260 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:03.456016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a076a-5b84-4228-b452-5e5ed069d314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:03.456877 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a076a-5b84-4228-b452-5e5ed069d314 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:03.457206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a076a-5b84-4228-b452-5e5ed069d314 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:03.457568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a076a-5b84-4228-b452-5e5ed069d314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:03.469699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a076a-5b84-4228-b452-5e5ed069d314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:03.471779 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a076a-5b84-4228-b452-5e5ed069d314 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:03.484425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1268/2522] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:09.009704 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:12.643365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61b4f5-288c-4682-96c9-89b942c08439 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:12.721610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61b4f5-288c-4682-96c9-89b942c08439 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:12.727150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:12.728151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:12.728628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:12.728912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] PATH_INFO: `/v3/users/eea05e0acc09483f8b5a97ca5ce8d5ea` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:12.730107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea05e0acc09483f8b5a97ca5ce8d5ea)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:12.732392 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:12.786226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea05e0acc09483f8b5a97ca5ce8d5ea'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:12.797850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea05e0acc09483f8b5a97ca5ce8d5ea'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:12.798629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea05e0acc09483f8b5a97ca5ce8d5ea'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:12.799226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea05e0acc09483f8b5a97ca5ce8d5ea'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:12.805814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea05e0acc09483f8b5a97ca5ce8d5ea'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:12.820014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1255/2523] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:34:12 2022] DELETE /identity/v3/users/eea05e0acc09483f8b5a97ca5ce8d5ea => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:12.825297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352c26-230d-4005-b5ad-5e1543c490e8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:12.825828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352c26-230d-4005-b5ad-5e1543c490e8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:12.825936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352c26-230d-4005-b5ad-5e1543c490e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:12.844418 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-56352c26-230d-4005-b5ad-5e1543c490e8 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:12.845610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-56352c26-230d-4005-b5ad-5e1543c490e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:12.921640 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1269/2524] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:12.933457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:12.935393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:13.011966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2339d-58f7-45e2-8288-de4a5f9532ed admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:13.016092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfa245-bf8c-4847-bd4a-96a93e0191ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:13.016754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfa245-bf8c-4847-bd4a-96a93e0191ae service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:13.017035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfa245-bf8c-4847-bd4a-96a93e0191ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:13.017310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfa245-bf8c-4847-bd4a-96a93e0191ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:13.028597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfa245-bf8c-4847-bd4a-96a93e0191ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:13.030199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfa245-bf8c-4847-bd4a-96a93e0191ae service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:13.042382 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1256/2525] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:13.386150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56352c26-230d-4005-b5ad-5e1543c490e8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:13.389103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-56352c26-230d-4005-b5ad-5e1543c490e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:13.406621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56352c26-230d-4005-b5ad-5e1543c490e8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:13.413194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:13.414069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:13.414343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:13.414639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] PATH_INFO: `/v3/projects/0622aec4ed3d4381a6911baf5bce8e8d` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:13.416438 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] RBAC: Authorizing `identity:delete_project(project_id=0622aec4ed3d4381a6911baf5bce8e8d)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:13.418460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:13.428203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:13.488106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0622aec4ed3d4381a6911baf5bce8e8d', 'request_id': 'req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:13.497421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0622aec4ed3d4381a6911baf5bce8e8d', 'request_id': 'req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:13.505567 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0622aec4ed3d4381a6911baf5bce8e8d', 'request_id': 'req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:13.518213 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] The token cache is being invalidate because project 0622aec4ed3d4381a6911baf5bce8e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:34:13.518498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:13.519263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:13.521011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1270/2526] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:34:13 2022] DELETE /identity/v3/projects/0622aec4ed3d4381a6911baf5bce8e8d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:13.525447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfa245-bf8c-4847-bd4a-96a93e0191ae service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:13.537154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfa245-bf8c-4847-bd4a-96a93e0191ae service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:13.561764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:13.620028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00ae36-14ab-4dad-aa0e-6bbf67a3b920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:13.620945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00ae36-14ab-4dad-aa0e-6bbf67a3b920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:13.621196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00ae36-14ab-4dad-aa0e-6bbf67a3b920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:13.621475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00ae36-14ab-4dad-aa0e-6bbf67a3b920 admin admin] PATH_INFO: `/v3/users/0a71051dc5bf4c449e66065ae2aa370a` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:13.622769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00ae36-14ab-4dad-aa0e-6bbf67a3b920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a71051dc5bf4c449e66065ae2aa370a)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:13.624697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00ae36-14ab-4dad-aa0e-6bbf67a3b920 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:13.689605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-db00ae36-14ab-4dad-aa0e-6bbf67a3b920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a71051dc5bf4c449e66065ae2aa370a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:13.709374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-db00ae36-14ab-4dad-aa0e-6bbf67a3b920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a71051dc5bf4c449e66065ae2aa370a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:13.710407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-db00ae36-14ab-4dad-aa0e-6bbf67a3b920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a71051dc5bf4c449e66065ae2aa370a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:13.711277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-db00ae36-14ab-4dad-aa0e-6bbf67a3b920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a71051dc5bf4c449e66065ae2aa370a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:13.719631 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-db00ae36-14ab-4dad-aa0e-6bbf67a3b920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a71051dc5bf4c449e66065ae2aa370a'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:13.738461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1257/2527] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:34:13 2022] DELETE /identity/v3/users/0a71051dc5bf4c449e66065ae2aa370a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:14.123445 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:14.125051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:14.148674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563aa5fc-7b29-4d80-b6b2-34fb9c3761da admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:14.219956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:14.220883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:14.221160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:14.221455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] PATH_INFO: `/v3/projects/c4b04072d2e34cc88754dad66731f45f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:14.222820 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b04072d2e34cc88754dad66731f45f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:14.224810 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:14.282755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b04072d2e34cc88754dad66731f45f', 'request_id': 'req-c34df1cc-d1b7-422a-948f-2ba863914912'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:14.291502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b04072d2e34cc88754dad66731f45f', 'request_id': 'req-c34df1cc-d1b7-422a-948f-2ba863914912'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:14.297393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b04072d2e34cc88754dad66731f45f', 'request_id': 'req-c34df1cc-d1b7-422a-948f-2ba863914912'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:14.311237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] The token cache is being invalidate because project c4b04072d2e34cc88754dad66731f45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:34:14.311527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:14.312310 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c34df1cc-d1b7-422a-948f-2ba863914912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:14.313997 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1271/2528] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:34:14 2022] DELETE /identity/v3/projects/c4b04072d2e34cc88754dad66731f45f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:14.320266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb830d9d-6efa-4b16-a964-5031c5d6d879 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:14.320454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb830d9d-6efa-4b16-a964-5031c5d6d879 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:14.320749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb830d9d-6efa-4b16-a964-5031c5d6d879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:14.341941 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-eb830d9d-6efa-4b16-a964-5031c5d6d879 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:14.343202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-eb830d9d-6efa-4b16-a964-5031c5d6d879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:14.426823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1258/2529] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:14.437816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc20999-3811-469c-82aa-09a5d148d766 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:14.438261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc20999-3811-469c-82aa-09a5d148d766 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:14.438542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc20999-3811-469c-82aa-09a5d148d766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:14.458239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dcc20999-3811-469c-82aa-09a5d148d766 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:14.459511 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dcc20999-3811-469c-82aa-09a5d148d766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:14.468747 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1272/2530] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:14.473228 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb830d9d-6efa-4b16-a964-5031c5d6d879 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:14.484752 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb830d9d-6efa-4b16-a964-5031c5d6d879 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:14.489917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ce136-8e93-450b-8229-f8f11fc0822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:14.490607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ce136-8e93-450b-8229-f8f11fc0822f admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:14.490892 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ce136-8e93-450b-8229-f8f11fc0822f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:14.491164 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ce136-8e93-450b-8229-f8f11fc0822f admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:14.491918 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ce136-8e93-450b-8229-f8f11fc0822f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:34:14.492224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ce136-8e93-450b-8229-f8f11fc0822f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:14.494085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ce136-8e93-450b-8229-f8f11fc0822f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:14.506511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1259/2531] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:34:14.511582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc20999-3811-469c-82aa-09a5d148d766 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:14.521330 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc20999-3811-469c-82aa-09a5d148d766 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:14.526613 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fbd06-e8ad-4da6-aeac-f9a9dae7b45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:14.527307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fbd06-e8ad-4da6-aeac-f9a9dae7b45d admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:14.527610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fbd06-e8ad-4da6-aeac-f9a9dae7b45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:14.527905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fbd06-e8ad-4da6-aeac-f9a9dae7b45d admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:14.528646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fbd06-e8ad-4da6-aeac-f9a9dae7b45d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:14.530693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fbd06-e8ad-4da6-aeac-f9a9dae7b45d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:14.546279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:14.564003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1273/2532] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:34:14 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:34:14.565448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ce136-8e93-450b-8229-f8f11fc0822f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:14.575825 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ce136-8e93-450b-8229-f8f11fc0822f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:14.602418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:14.642837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dca5e-04e1-4144-b664-94e14d8a472c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:14.643544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dca5e-04e1-4144-b664-94e14d8a472c admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:14.643809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dca5e-04e1-4144-b664-94e14d8a472c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:14.644415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dca5e-04e1-4144-b664-94e14d8a472c admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:14.645131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312dca5e-04e1-4144-b664-94e14d8a472c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:14.647164 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312dca5e-04e1-4144-b664-94e14d8a472c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:14.673532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1260/2533] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:34:14 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:34:14.676303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fbd06-e8ad-4da6-aeac-f9a9dae7b45d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:14.689343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fbd06-e8ad-4da6-aeac-f9a9dae7b45d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:14.697695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1d6a5-11f3-4842-a6d4-0c4a0c152fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:14.698383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d6a5-11f3-4842-a6d4-0c4a0c152fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:14.698699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d6a5-11f3-4842-a6d4-0c4a0c152fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:14.698973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d6a5-11f3-4842-a6d4-0c4a0c152fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:14.699708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1d6a5-11f3-4842-a6d4-0c4a0c152fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:14.701458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1d6a5-11f3-4842-a6d4-0c4a0c152fb1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:14.714056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1274/2534] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:34:14.718390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dca5e-04e1-4144-b664-94e14d8a472c admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:14.730336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dca5e-04e1-4144-b664-94e14d8a472c admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:14.738700 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f67e8-728d-457b-8c72-2c20ecfbc472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:14.739519 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f67e8-728d-457b-8c72-2c20ecfbc472 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:14.739956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f67e8-728d-457b-8c72-2c20ecfbc472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:14.740386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f67e8-728d-457b-8c72-2c20ecfbc472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:14.741505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f67e8-728d-457b-8c72-2c20ecfbc472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:14.743670 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f67e8-728d-457b-8c72-2c20ecfbc472 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:14.756027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1261/2535] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:34:14.761233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1d6a5-11f3-4842-a6d4-0c4a0c152fb1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:14.771983 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1d6a5-11f3-4842-a6d4-0c4a0c152fb1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:14.777453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88830425-f149-4252-8c57-e6e78d76e319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:14.778360 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88830425-f149-4252-8c57-e6e78d76e319 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:14.778667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88830425-f149-4252-8c57-e6e78d76e319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:14.778952 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88830425-f149-4252-8c57-e6e78d76e319 admin admin] PATH_INFO: `/v3/projects/c4065356e5474f888c572facb87b7894/users/cdc9d8f82cfd46ecb1e3105bf78c166c/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:14.797822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88830425-f149-4252-8c57-e6e78d76e319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4065356e5474f888c572facb87b7894, user_id=cdc9d8f82cfd46ecb1e3105bf78c166c, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:14.800633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88830425-f149-4252-8c57-e6e78d76e319 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:14.817731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1275/2536] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:34:14 2022] PUT /identity/v3/projects/c4065356e5474f888c572facb87b7894/users/cdc9d8f82cfd46ecb1e3105bf78c166c/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:14.829349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454c2d3-59f7-46be-88fe-d5566188827c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:14.829625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454c2d3-59f7-46be-88fe-d5566188827c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:14.829910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454c2d3-59f7-46be-88fe-d5566188827c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:14.860461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-1454c2d3-59f7-46be-88fe-d5566188827c None None] MFA Rules not processed for user `cdc9d8f82cfd46ecb1e3105bf78c166c`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:14.861984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-1454c2d3-59f7-46be-88fe-d5566188827c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:14.950959 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1262/2537] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:14.964541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88830425-f149-4252-8c57-e6e78d76e319 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:14.967266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-88830425-f149-4252-8c57-e6e78d76e319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:15.061652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88830425-f149-4252-8c57-e6e78d76e319 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:15.065513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e03eed-6387-49eb-9431-e3df495c0fca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e850be24df24bcf972e6e31002014cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:15.066196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03eed-6387-49eb-9431-e3df495c0fca service glance] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:15.066527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03eed-6387-49eb-9431-e3df495c0fca service glance] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:15.066819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03eed-6387-49eb-9431-e3df495c0fca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:15.077486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e03eed-6387-49eb-9431-e3df495c0fca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:15.078964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e03eed-6387-49eb-9431-e3df495c0fca service glance] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:15.088797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1276/2538] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:24.944290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:25.084275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:38.106601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454c2d3-59f7-46be-88fe-d5566188827c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:38.175236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454c2d3-59f7-46be-88fe-d5566188827c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:38.179864 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:38.180746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:38.181022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:38.181293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] PATH_INFO: `/v3/users/cdc9d8f82cfd46ecb1e3105bf78c166c` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:38.182945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc9d8f82cfd46ecb1e3105bf78c166c)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:38.185417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:38.237952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc9d8f82cfd46ecb1e3105bf78c166c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:38.250972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc9d8f82cfd46ecb1e3105bf78c166c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:38.251814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc9d8f82cfd46ecb1e3105bf78c166c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:38.252590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc9d8f82cfd46ecb1e3105bf78c166c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:38.260453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc9d8f82cfd46ecb1e3105bf78c166c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:38.275440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1263/2539] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:34:38 2022] DELETE /identity/v3/users/cdc9d8f82cfd46ecb1e3105bf78c166c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:38.280961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578c789-1b97-46ec-8bec-70ad38179220 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:38.281521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578c789-1b97-46ec-8bec-70ad38179220 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:38.281521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578c789-1b97-46ec-8bec-70ad38179220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:38.301828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2578c789-1b97-46ec-8bec-70ad38179220 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:38.303060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2578c789-1b97-46ec-8bec-70ad38179220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:38.368499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1277/2540] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:38.378461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:38.379956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:38.440263 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761f972-870d-4ad0-a4f0-ec15b25411af admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:38.445185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995485e4-5352-4a22-93bc-7b1f9c4de3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:38.445847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995485e4-5352-4a22-93bc-7b1f9c4de3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:38.446446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995485e4-5352-4a22-93bc-7b1f9c4de3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:38.446446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995485e4-5352-4a22-93bc-7b1f9c4de3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:38.457694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995485e4-5352-4a22-93bc-7b1f9c4de3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:38.459182 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995485e4-5352-4a22-93bc-7b1f9c4de3f5 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:38.470840 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1264/2541] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:38.749012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578c789-1b97-46ec-8bec-70ad38179220 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:38.750826 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2578c789-1b97-46ec-8bec-70ad38179220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:38.765390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578c789-1b97-46ec-8bec-70ad38179220 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:38.770619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:38.771537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:38.771818 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:38.772167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] PATH_INFO: `/v3/projects/c4065356e5474f888c572facb87b7894` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:38.773417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4065356e5474f888c572facb87b7894)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:38.775454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:38.825952 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4065356e5474f888c572facb87b7894', 'request_id': 'req-8dd582ce-dd11-4661-9213-063abb5b9560'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:38.834078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4065356e5474f888c572facb87b7894', 'request_id': 'req-8dd582ce-dd11-4661-9213-063abb5b9560'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:38.839976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4065356e5474f888c572facb87b7894', 'request_id': 'req-8dd582ce-dd11-4661-9213-063abb5b9560'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:38.852133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] The token cache is being invalidate because project c4065356e5474f888c572facb87b7894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:34:38.852415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:38.853129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8dd582ce-dd11-4661-9213-063abb5b9560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:38.854837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1278/2542] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:34:38 2022] DELETE /identity/v3/projects/c4065356e5474f888c572facb87b7894 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:38.860170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3a3a4-909e-427d-bb23-59adb8ae2554 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:38.860458 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3a3a4-909e-427d-bb23-59adb8ae2554 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:38.860736 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3a3a4-909e-427d-bb23-59adb8ae2554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:38.879088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-7ec3a3a4-909e-427d-bb23-59adb8ae2554 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:38.880337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7ec3a3a4-909e-427d-bb23-59adb8ae2554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:38.899970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:38.947851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1265/2543] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:38.956350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b8fd5-3a6d-4b31-99a4-a647dc8fd82f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:38.956618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b8fd5-3a6d-4b31-99a4-a647dc8fd82f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:38.956884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b8fd5-3a6d-4b31-99a4-a647dc8fd82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:38.976177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-077b8fd5-3a6d-4b31-99a4-a647dc8fd82f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:38.977368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-077b8fd5-3a6d-4b31-99a4-a647dc8fd82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:38.986614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1279/2544] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:38.991056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3a3a4-909e-427d-bb23-59adb8ae2554 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:39.002245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3a3a4-909e-427d-bb23-59adb8ae2554 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:39.007080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c7bae-446f-4f13-81e0-f32a77dc6601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:39.007779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c7bae-446f-4f13-81e0-f32a77dc6601 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.008062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c7bae-446f-4f13-81e0-f32a77dc6601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.008340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c7bae-446f-4f13-81e0-f32a77dc6601 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.009079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c7bae-446f-4f13-81e0-f32a77dc6601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:34:39.009389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c7bae-446f-4f13-81e0-f32a77dc6601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:39.011222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c7bae-446f-4f13-81e0-f32a77dc6601 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:39.023917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1266/2545] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:34:39.028712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b8fd5-3a6d-4b31-99a4-a647dc8fd82f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:39.038485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b8fd5-3a6d-4b31-99a4-a647dc8fd82f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:39.044813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9cb64-b0df-4143-a70c-cd9fb4cf9ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:39.045457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9cb64-b0df-4143-a70c-cd9fb4cf9ccf admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.045738 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9cb64-b0df-4143-a70c-cd9fb4cf9ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.046005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9cb64-b0df-4143-a70c-cd9fb4cf9ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.046746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9cb64-b0df-4143-a70c-cd9fb4cf9ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:39.048656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9cb64-b0df-4143-a70c-cd9fb4cf9ccf admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:39.078493 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1280/2546] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:34:39 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:34:39.082669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c7bae-446f-4f13-81e0-f32a77dc6601 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:39.094274 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c7bae-446f-4f13-81e0-f32a77dc6601 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:39.156028 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067e041-b614-4071-9949-adc4ccefbdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:39.156776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067e041-b614-4071-9949-adc4ccefbdf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.156983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067e041-b614-4071-9949-adc4ccefbdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.157260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067e041-b614-4071-9949-adc4ccefbdf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.157977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067e041-b614-4071-9949-adc4ccefbdf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:39.159924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067e041-b614-4071-9949-adc4ccefbdf6 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:39.180963 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1267/2547] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:34:39 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:34:39.186349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9cb64-b0df-4143-a70c-cd9fb4cf9ccf admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:39.196905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9cb64-b0df-4143-a70c-cd9fb4cf9ccf admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:39.202675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea485fe6-49e6-4935-9413-a7c2cdfe9818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:39.203333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea485fe6-49e6-4935-9413-a7c2cdfe9818 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.203627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea485fe6-49e6-4935-9413-a7c2cdfe9818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.203896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea485fe6-49e6-4935-9413-a7c2cdfe9818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.204626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea485fe6-49e6-4935-9413-a7c2cdfe9818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:39.206266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea485fe6-49e6-4935-9413-a7c2cdfe9818 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:39.217468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1281/2548] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:34:39.224416 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067e041-b614-4071-9949-adc4ccefbdf6 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:39.240866 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067e041-b614-4071-9949-adc4ccefbdf6 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:39.248495 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d19f63-5963-4c01-a464-83eafb4a02d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:39.249170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19f63-5963-4c01-a464-83eafb4a02d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.249451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19f63-5963-4c01-a464-83eafb4a02d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.249724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19f63-5963-4c01-a464-83eafb4a02d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.250456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d19f63-5963-4c01-a464-83eafb4a02d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:39.252118 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d19f63-5963-4c01-a464-83eafb4a02d7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:39.263187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1268/2549] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:34:39.267618 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea485fe6-49e6-4935-9413-a7c2cdfe9818 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:39.278295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea485fe6-49e6-4935-9413-a7c2cdfe9818 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:39.283183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9c78b-2a52-4f8b-a795-f08688ee2b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:39.284071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c78b-2a52-4f8b-a795-f08688ee2b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.284354 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c78b-2a52-4f8b-a795-f08688ee2b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.284632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c78b-2a52-4f8b-a795-f08688ee2b19 admin admin] PATH_INFO: `/v3/projects/b2af7910f7d941ac9fe3ccd39719e4b1/users/597ea771fed3404781ac3b7fa4e4c1d4/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.300274 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9c78b-2a52-4f8b-a795-f08688ee2b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2af7910f7d941ac9fe3ccd39719e4b1, user_id=597ea771fed3404781ac3b7fa4e4c1d4, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:39.302998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9c78b-2a52-4f8b-a795-f08688ee2b19 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:39.317933 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1282/2550] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:34:39 2022] PUT /identity/v3/projects/b2af7910f7d941ac9fe3ccd39719e4b1/users/597ea771fed3404781ac3b7fa4e4c1d4/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:39.329212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58d49c-33be-425e-98cb-47abd31b45c4 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.329500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58d49c-33be-425e-98cb-47abd31b45c4 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.329774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58d49c-33be-425e-98cb-47abd31b45c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.358793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-eb58d49c-33be-425e-98cb-47abd31b45c4 None None] MFA Rules not processed for user `597ea771fed3404781ac3b7fa4e4c1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:39.360009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-eb58d49c-33be-425e-98cb-47abd31b45c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:39.448399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1269/2551] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:39.454668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9c78b-2a52-4f8b-a795-f08688ee2b19 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:39.475719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:39.521558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9c78b-2a52-4f8b-a795-f08688ee2b19 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:39.526930 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a75326-043a-4b89-a2ef-ce82846ea58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:39.527600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a75326-043a-4b89-a2ef-ce82846ea58d admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.527892 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a75326-043a-4b89-a2ef-ce82846ea58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.528163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a75326-043a-4b89-a2ef-ce82846ea58d admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.528865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a75326-043a-4b89-a2ef-ce82846ea58d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:39.531695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a75326-043a-4b89-a2ef-ce82846ea58d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:39.561654 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1283/2552] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:34:39 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:34:39.565735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58d49c-33be-425e-98cb-47abd31b45c4 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:39.577740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58d49c-33be-425e-98cb-47abd31b45c4 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:39.592510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:39.644955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee3439-a8b3-4882-8cd8-fb2177dc037d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:39.645428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee3439-a8b3-4882-8cd8-fb2177dc037d admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.645697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee3439-a8b3-4882-8cd8-fb2177dc037d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.645972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee3439-a8b3-4882-8cd8-fb2177dc037d admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.646695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee3439-a8b3-4882-8cd8-fb2177dc037d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:39.648672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee3439-a8b3-4882-8cd8-fb2177dc037d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:39.672310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1270/2553] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:34:39 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:34:39.676581 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a75326-043a-4b89-a2ef-ce82846ea58d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:39.690885 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a75326-043a-4b89-a2ef-ce82846ea58d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:39.695949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9765ba2-593c-4228-868c-7c813f3294cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:39.696628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9765ba2-593c-4228-868c-7c813f3294cd admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.696895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9765ba2-593c-4228-868c-7c813f3294cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.697168 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9765ba2-593c-4228-868c-7c813f3294cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.697918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9765ba2-593c-4228-868c-7c813f3294cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:39.699851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9765ba2-593c-4228-868c-7c813f3294cd admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:39.712293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1284/2554] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:34:39.716550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee3439-a8b3-4882-8cd8-fb2177dc037d admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:39.726734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee3439-a8b3-4882-8cd8-fb2177dc037d admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:39.731847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381a08a-3a18-4d0e-8bab-1bb4c5e0d239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:39.732510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381a08a-3a18-4d0e-8bab-1bb4c5e0d239 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.732801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381a08a-3a18-4d0e-8bab-1bb4c5e0d239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.733082 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381a08a-3a18-4d0e-8bab-1bb4c5e0d239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.733824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381a08a-3a18-4d0e-8bab-1bb4c5e0d239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:39.735482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381a08a-3a18-4d0e-8bab-1bb4c5e0d239 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:39.747048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1271/2555] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:34:39.751495 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9765ba2-593c-4228-868c-7c813f3294cd admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:39.763299 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9765ba2-593c-4228-868c-7c813f3294cd admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:39.775794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e4df5-c3e4-460b-bf36-dbfaa4b1fe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:39.777584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e4df5-c3e4-460b-bf36-dbfaa4b1fe0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.777584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e4df5-c3e4-460b-bf36-dbfaa4b1fe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.777584 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e4df5-c3e4-460b-bf36-dbfaa4b1fe0e admin admin] PATH_INFO: `/v3/projects/a8568e0d64f14414aaeb92e21ad11e25/users/216e114193fa4011a22a40474c8af57c/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.793487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e4df5-c3e4-460b-bf36-dbfaa4b1fe0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8568e0d64f14414aaeb92e21ad11e25, user_id=216e114193fa4011a22a40474c8af57c, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:39.796291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e4df5-c3e4-460b-bf36-dbfaa4b1fe0e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:39.811025 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1285/2556] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:34:39 2022] PUT /identity/v3/projects/a8568e0d64f14414aaeb92e21ad11e25/users/216e114193fa4011a22a40474c8af57c/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:34:39.821482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a313b-c76c-47b8-8438-fea133a35db1 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:39.821775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a313b-c76c-47b8-8438-fea133a35db1 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:39.822020 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a313b-c76c-47b8-8438-fea133a35db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:39.850599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-5c9a313b-c76c-47b8-8438-fea133a35db1 None None] MFA Rules not processed for user `216e114193fa4011a22a40474c8af57c`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:39.851850 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5c9a313b-c76c-47b8-8438-fea133a35db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:39.924429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1272/2557] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:39.932399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e4df5-c3e4-460b-bf36-dbfaa4b1fe0e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:40.001479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e4df5-c3e4-460b-bf36-dbfaa4b1fe0e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:40.008569 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efdcd0-030f-493d-a95d-f757ccc4e6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:40.009263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efdcd0-030f-493d-a95d-f757ccc4e6dd admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:40.009518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efdcd0-030f-493d-a95d-f757ccc4e6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:40.009784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efdcd0-030f-493d-a95d-f757ccc4e6dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:40.010478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4efdcd0-030f-493d-a95d-f757ccc4e6dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:40.013137 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4efdcd0-030f-493d-a95d-f757ccc4e6dd admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:40.054637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1286/2558] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:34:39 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:34:40.056196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a313b-c76c-47b8-8438-fea133a35db1 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:40.072110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a313b-c76c-47b8-8438-fea133a35db1 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:40.133317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69229a08-164d-407e-ad5d-52cd11412b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:40.134009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69229a08-164d-407e-ad5d-52cd11412b5d admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:40.134290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69229a08-164d-407e-ad5d-52cd11412b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:40.134591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69229a08-164d-407e-ad5d-52cd11412b5d admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:40.135325 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69229a08-164d-407e-ad5d-52cd11412b5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:40.137553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69229a08-164d-407e-ad5d-52cd11412b5d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:40.160338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1273/2559] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:34:40 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:34:40.164447 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efdcd0-030f-493d-a95d-f757ccc4e6dd admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:40.175451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efdcd0-030f-493d-a95d-f757ccc4e6dd admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:40.181038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d2ba8-5304-408f-982d-f59223344a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:40.181706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d2ba8-5304-408f-982d-f59223344a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:40.182449 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d2ba8-5304-408f-982d-f59223344a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:40.182753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d2ba8-5304-408f-982d-f59223344a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:40.183993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d2ba8-5304-408f-982d-f59223344a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:40.185614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d2ba8-5304-408f-982d-f59223344a38 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:40.197365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1287/2560] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 10:34:40.202513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69229a08-164d-407e-ad5d-52cd11412b5d admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:40.213600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69229a08-164d-407e-ad5d-52cd11412b5d admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:40.219800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa0463-499d-45bd-8fe1-69b368de72e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:40.220718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa0463-499d-45bd-8fe1-69b368de72e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:40.221005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa0463-499d-45bd-8fe1-69b368de72e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:40.221290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa0463-499d-45bd-8fe1-69b368de72e2 admin admin] PATH_INFO: `/v3/projects/e492e78757084a63a175208b1df124b3/users/a34357c101694209a5d2b8b8059cd0af/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:40.235752 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:40.239816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa0463-499d-45bd-8fe1-69b368de72e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e492e78757084a63a175208b1df124b3, user_id=a34357c101694209a5d2b8b8059cd0af, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:40.243694 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa0463-499d-45bd-8fe1-69b368de72e2 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:40.259566 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1274/2561] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:34:40 2022] PUT /identity/v3/projects/e492e78757084a63a175208b1df124b3/users/a34357c101694209a5d2b8b8059cd0af/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:40.263888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d2ba8-5304-408f-982d-f59223344a38 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:40.274760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d2ba8-5304-408f-982d-f59223344a38 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:40.325211 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:40.341931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407fd61-06db-4641-9a32-ee7737f9936e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:40.342627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407fd61-06db-4641-9a32-ee7737f9936e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:40.342900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407fd61-06db-4641-9a32-ee7737f9936e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:40.343178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407fd61-06db-4641-9a32-ee7737f9936e admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:40.343903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7407fd61-06db-4641-9a32-ee7737f9936e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:40.345527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7407fd61-06db-4641-9a32-ee7737f9936e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:40.357982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1288/2562] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:34:40.362514 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa0463-499d-45bd-8fe1-69b368de72e2 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:40.373722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa0463-499d-45bd-8fe1-69b368de72e2 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:40.379553 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e576c-53ca-4a49-bfdc-27061c11fbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:40.380465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e576c-53ca-4a49-bfdc-27061c11fbf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:40.380742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e576c-53ca-4a49-bfdc-27061c11fbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:40.381015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e576c-53ca-4a49-bfdc-27061c11fbf7 admin admin] PATH_INFO: `/v3/projects/e492e78757084a63a175208b1df124b3/users/a34357c101694209a5d2b8b8059cd0af/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:40.383214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e576c-53ca-4a49-bfdc-27061c11fbf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e492e78757084a63a175208b1df124b3, user_id=a34357c101694209a5d2b8b8059cd0af, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:40.385893 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e576c-53ca-4a49-bfdc-27061c11fbf7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:40.398299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1275/2563] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:34:40 2022] PUT /identity/v3/projects/e492e78757084a63a175208b1df124b3/users/a34357c101694209a5d2b8b8059cd0af/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:40.411050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a98eb4-a91c-4674-8920-dfe679bfdb8e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:40.411295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a98eb4-a91c-4674-8920-dfe679bfdb8e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:40.411605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a98eb4-a91c-4674-8920-dfe679bfdb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:40.440539 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-10a98eb4-a91c-4674-8920-dfe679bfdb8e None None] MFA Rules not processed for user `a34357c101694209a5d2b8b8059cd0af`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:40.441776 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-10a98eb4-a91c-4674-8920-dfe679bfdb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:40.520669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1289/2564] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:40.532681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e576c-53ca-4a49-bfdc-27061c11fbf7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:40.534508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ba0e576c-53ca-4a49-bfdc-27061c11fbf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:40.597314 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e576c-53ca-4a49-bfdc-27061c11fbf7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:40.601136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89fba4-07ee-4fa6-8ec2-4eca73b4503a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:40.601801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89fba4-07ee-4fa6-8ec2-4eca73b4503a service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:40.602080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89fba4-07ee-4fa6-8ec2-4eca73b4503a service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:40.602356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89fba4-07ee-4fa6-8ec2-4eca73b4503a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:40.665746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89fba4-07ee-4fa6-8ec2-4eca73b4503a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:40.667284 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89fba4-07ee-4fa6-8ec2-4eca73b4503a service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:40.677405 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1276/2565] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:40.749917 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a98eb4-a91c-4674-8920-dfe679bfdb8e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:40.751570 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-10a98eb4-a91c-4674-8920-dfe679bfdb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:40.765294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a98eb4-a91c-4674-8920-dfe679bfdb8e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:40.769487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb250ee-a9c7-45f9-b7c6-2824683fae37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:40.770142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb250ee-a9c7-45f9-b7c6-2824683fae37 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:40.770421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb250ee-a9c7-45f9-b7c6-2824683fae37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:40.770728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb250ee-a9c7-45f9-b7c6-2824683fae37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:40.772586 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3eb250ee-a9c7-45f9-b7c6-2824683fae37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:40.840962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb250ee-a9c7-45f9-b7c6-2824683fae37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:40.843361 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb250ee-a9c7-45f9-b7c6-2824683fae37 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:40.855457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1290/2566] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:42.223582 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89fba4-07ee-4fa6-8ec2-4eca73b4503a service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:42.233402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89fba4-07ee-4fa6-8ec2-4eca73b4503a service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:42.237983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc041c8-6242-4d55-93f7-0d80c2c70f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:42.238678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc041c8-6242-4d55-93f7-0d80c2c70f82 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:42.238958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc041c8-6242-4d55-93f7-0d80c2c70f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:42.239224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc041c8-6242-4d55-93f7-0d80c2c70f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:42.302644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc041c8-6242-4d55-93f7-0d80c2c70f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:42.304523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc041c8-6242-4d55-93f7-0d80c2c70f82 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:42.315387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1277/2567] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:42.892681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb250ee-a9c7-45f9-b7c6-2824683fae37 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:42.903460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb250ee-a9c7-45f9-b7c6-2824683fae37 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:42.907675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d367013-d6a6-434c-a0cc-41506a63e107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:42.908331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367013-d6a6-434c-a0cc-41506a63e107 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:42.908607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367013-d6a6-434c-a0cc-41506a63e107 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:42.908875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367013-d6a6-434c-a0cc-41506a63e107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:42.920295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d367013-d6a6-434c-a0cc-41506a63e107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:42.921758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d367013-d6a6-434c-a0cc-41506a63e107 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:42.933013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1291/2568] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:46.934664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc041c8-6242-4d55-93f7-0d80c2c70f82 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:46.984342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:47.022148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc041c8-6242-4d55-93f7-0d80c2c70f82 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:47.027377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:47.028947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:47.029221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:47.029488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] PATH_INFO: `/v3/users/597ea771fed3404781ac3b7fa4e4c1d4` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:47.030799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=597ea771fed3404781ac3b7fa4e4c1d4)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:47.032734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:47.094824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '597ea771fed3404781ac3b7fa4e4c1d4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.109276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '597ea771fed3404781ac3b7fa4e4c1d4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.110161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '597ea771fed3404781ac3b7fa4e4c1d4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.110948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '597ea771fed3404781ac3b7fa4e4c1d4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.118197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '597ea771fed3404781ac3b7fa4e4c1d4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.132689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1278/2569] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:34:46 2022] DELETE /identity/v3/users/597ea771fed3404781ac3b7fa4e4c1d4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:47.137335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ce82c-6fb6-4304-9150-07ee1e23e097 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:47.137819 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ce82c-6fb6-4304-9150-07ee1e23e097 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:47.137952 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ce82c-6fb6-4304-9150-07ee1e23e097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:47.156440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c50ce82c-6fb6-4304-9150-07ee1e23e097 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:47.157685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c50ce82c-6fb6-4304-9150-07ee1e23e097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:47.207043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:47.225914 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1292/2570] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:47.237038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:47.239193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:47.329925 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26267416-bcb4-4268-8265-dfc264f6b408 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:47.334177 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841b3f1-cc62-4ec7-bc25-a0b5e60e7124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:47.334866 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841b3f1-cc62-4ec7-bc25-a0b5e60e7124 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:47.335148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841b3f1-cc62-4ec7-bc25-a0b5e60e7124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:47.335421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841b3f1-cc62-4ec7-bc25-a0b5e60e7124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:47.346076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841b3f1-cc62-4ec7-bc25-a0b5e60e7124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:47.347563 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841b3f1-cc62-4ec7-bc25-a0b5e60e7124 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:47.357849 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1279/2571] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:47.661021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ce82c-6fb6-4304-9150-07ee1e23e097 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:47.662606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c50ce82c-6fb6-4304-9150-07ee1e23e097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:47.678413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ce82c-6fb6-4304-9150-07ee1e23e097 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:47.686218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:47.687701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:47.688200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:47.688606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] PATH_INFO: `/v3/projects/b2af7910f7d941ac9fe3ccd39719e4b1` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:47.691852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2af7910f7d941ac9fe3ccd39719e4b1)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:47.695365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:47.771817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2af7910f7d941ac9fe3ccd39719e4b1', 'request_id': 'req-f5c3abf8-7730-4c07-aaec-f82d64f781d8'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.783017 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2af7910f7d941ac9fe3ccd39719e4b1', 'request_id': 'req-f5c3abf8-7730-4c07-aaec-f82d64f781d8'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.789343 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2af7910f7d941ac9fe3ccd39719e4b1', 'request_id': 'req-f5c3abf8-7730-4c07-aaec-f82d64f781d8'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.802661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] The token cache is being invalidate because project b2af7910f7d941ac9fe3ccd39719e4b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:34:47.802975 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.803607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.805198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1293/2572] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:34:47 2022] DELETE /identity/v3/projects/b2af7910f7d941ac9fe3ccd39719e4b1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:34:47.819568 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841b3f1-cc62-4ec7-bc25-a0b5e60e7124 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:47.834517 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841b3f1-cc62-4ec7-bc25-a0b5e60e7124 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:47.835099 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:47.836902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:47.871014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:47.906990 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:47.907953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:47.908235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:47.908511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] PATH_INFO: `/v3/users/216e114193fa4011a22a40474c8af57c` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:47.909702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=216e114193fa4011a22a40474c8af57c)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:47.911744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:47.931292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3abf8-7730-4c07-aaec-f82d64f781d8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:47.939476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4b6ad-7f5a-43c2-ad6a-6ccb53a96322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:47.940152 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4b6ad-7f5a-43c2-ad6a-6ccb53a96322 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:47.940433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4b6ad-7f5a-43c2-ad6a-6ccb53a96322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:47.940708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4b6ad-7f5a-43c2-ad6a-6ccb53a96322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:47.942757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8da4b6ad-7f5a-43c2-ad6a-6ccb53a96322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:47.958180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4b6ad-7f5a-43c2-ad6a-6ccb53a96322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:47.959695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4b6ad-7f5a-43c2-ad6a-6ccb53a96322 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:47.970996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1294/2573] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:47.972709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216e114193fa4011a22a40474c8af57c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.988437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216e114193fa4011a22a40474c8af57c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.989331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216e114193fa4011a22a40474c8af57c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.990040 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216e114193fa4011a22a40474c8af57c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:47.998687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216e114193fa4011a22a40474c8af57c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.018791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1280/2574] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:34:47 2022] DELETE /identity/v3/users/216e114193fa4011a22a40474c8af57c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:48.318674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4b6ad-7f5a-43c2-ad6a-6ccb53a96322 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:48.320277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8da4b6ad-7f5a-43c2-ad6a-6ccb53a96322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:48.333925 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4b6ad-7f5a-43c2-ad6a-6ccb53a96322 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:48.399661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:48.400565 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:48.400848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:48.401127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] PATH_INFO: `/v3/projects/a8568e0d64f14414aaeb92e21ad11e25` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:48.402629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8568e0d64f14414aaeb92e21ad11e25)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:48.405771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:48.429954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:48.461433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8568e0d64f14414aaeb92e21ad11e25', 'request_id': 'req-8cc99177-c848-4890-b2e9-e701e54be7a0'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.470472 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8568e0d64f14414aaeb92e21ad11e25', 'request_id': 'req-8cc99177-c848-4890-b2e9-e701e54be7a0'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.478949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8568e0d64f14414aaeb92e21ad11e25', 'request_id': 'req-8cc99177-c848-4890-b2e9-e701e54be7a0'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.479231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:48.480973 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:48.494132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] The token cache is being invalidate because project a8568e0d64f14414aaeb92e21ad11e25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:34:48.494658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.495219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.500912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1295/2575] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:34:48 2022] DELETE /identity/v3/projects/a8568e0d64f14414aaeb92e21ad11e25 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:48.502340 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:48.503763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:48.576074 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e6272-95b0-4926-82d4-836d551b77a4 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:48.593814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc99177-c848-4890-b2e9-e701e54be7a0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:48.599949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:48.600845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:48.601120 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:48.601387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] PATH_INFO: `/v3/users/a34357c101694209a5d2b8b8059cd0af` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:48.602696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a34357c101694209a5d2b8b8059cd0af)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:48.605063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:48.634057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:48.647487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44cff3-7edc-43ba-b8b3-523d92e19378 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:48.648366 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44cff3-7edc-43ba-b8b3-523d92e19378 service placement] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:48.648651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44cff3-7edc-43ba-b8b3-523d92e19378 service placement] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:48.648927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44cff3-7edc-43ba-b8b3-523d92e19378 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:48.651017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2f44cff3-7edc-43ba-b8b3-523d92e19378 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:48.668471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44cff3-7edc-43ba-b8b3-523d92e19378 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:48.670096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44cff3-7edc-43ba-b8b3-523d92e19378 service placement] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:48.679287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a34357c101694209a5d2b8b8059cd0af'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.680641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1281/2576] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:48.681540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44cff3-7edc-43ba-b8b3-523d92e19378 service placement] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:48.683241 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2f44cff3-7edc-43ba-b8b3-523d92e19378 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:48.693351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a34357c101694209a5d2b8b8059cd0af'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.694183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a34357c101694209a5d2b8b8059cd0af'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.699058 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44cff3-7edc-43ba-b8b3-523d92e19378 service placement] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:48.702968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a34357c101694209a5d2b8b8059cd0af'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.703665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a34357c101694209a5d2b8b8059cd0af'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.704759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:48.705656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:48.705933 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:48.706204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] PATH_INFO: `/v3/users/43964a0e6b48487bb90c5356d312b60d` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:48.707684 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43964a0e6b48487bb90c5356d312b60d)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:48.710106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:48.719564 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1296/2577] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:34:48 2022] DELETE /identity/v3/users/a34357c101694209a5d2b8b8059cd0af => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:34:48.770067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43964a0e6b48487bb90c5356d312b60d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.788929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43964a0e6b48487bb90c5356d312b60d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.790083 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43964a0e6b48487bb90c5356d312b60d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.791314 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43964a0e6b48487bb90c5356d312b60d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.800251 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43964a0e6b48487bb90c5356d312b60d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:48.815531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1282/2578] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:34:48 2022] DELETE /identity/v3/users/43964a0e6b48487bb90c5356d312b60d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:49.050757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.052634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:49.075124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.078777 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:49.114404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:49.165915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269857a-0eed-479a-979c-230b074c559a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.171782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.172690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.172969 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.173241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] PATH_INFO: `/v3/projects/e492e78757084a63a175208b1df124b3` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.174527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e492e78757084a63a175208b1df124b3)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.176553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.177606 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ced989-7c07-4146-9ba2-3003b9a2f832 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.183173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.184210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.184799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.184928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] PATH_INFO: `/v3/projects/4779579fbd6f42dcb6ad98e752ef741c` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.186066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4779579fbd6f42dcb6ad98e752ef741c)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.188049 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.229823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e492e78757084a63a175208b1df124b3', 'request_id': 'req-71e203ba-f574-41e8-ac78-13c358fff47b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:49.239941 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e492e78757084a63a175208b1df124b3', 'request_id': 'req-71e203ba-f574-41e8-ac78-13c358fff47b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:49.240179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4779579fbd6f42dcb6ad98e752ef741c', 'request_id': 'req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:49.248485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e492e78757084a63a175208b1df124b3', 'request_id': 'req-71e203ba-f574-41e8-ac78-13c358fff47b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:49.248899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4779579fbd6f42dcb6ad98e752ef741c', 'request_id': 'req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:49.252148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:49.256924 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4779579fbd6f42dcb6ad98e752ef741c', 'request_id': 'req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:49.263756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] The token cache is being invalidate because project e492e78757084a63a175208b1df124b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:34:49.264164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:49.264973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-71e203ba-f574-41e8-ac78-13c358fff47b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:49.266587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1297/2579] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:34:49 2022] DELETE /identity/v3/projects/e492e78757084a63a175208b1df124b3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:49.273548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7983e-b624-4538-8fcd-847e0c1b9b21 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.273827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7983e-b624-4538-8fcd-847e0c1b9b21 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.274085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7983e-b624-4538-8fcd-847e0c1b9b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.276122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] The token cache is being invalidate because project 4779579fbd6f42dcb6ad98e752ef741c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:34:49.276414 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:49.277111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ddedce92-4b63-4d40-b5aa-3ad640e00cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:34:49.279743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1283/2580] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:34:49 2022] DELETE /identity/v3/projects/4779579fbd6f42dcb6ad98e752ef741c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:34:49.286265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d720338-e49d-4571-b8d3-756732b7a4a9 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.286550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d720338-e49d-4571-b8d3-756732b7a4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.286853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d720338-e49d-4571-b8d3-756732b7a4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.299293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-17d7983e-b624-4538-8fcd-847e0c1b9b21 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:49.300685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-17d7983e-b624-4538-8fcd-847e0c1b9b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:49.308000 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-4d720338-e49d-4571-b8d3-756732b7a4a9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:49.309762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4d720338-e49d-4571-b8d3-756732b7a4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:49.373816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1298/2581] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:34:49.377316 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1284/2582] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:34:49.382040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b46808-2cfe-4762-841a-d222ac6f4aa6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.382338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b46808-2cfe-4762-841a-d222ac6f4aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.382599 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b46808-2cfe-4762-841a-d222ac6f4aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.385386 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597814db-c120-4f99-9e6d-d7c88e891044 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.385818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597814db-c120-4f99-9e6d-d7c88e891044 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.386133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597814db-c120-4f99-9e6d-d7c88e891044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.402801 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-26b46808-2cfe-4762-841a-d222ac6f4aa6 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:49.404137 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-26b46808-2cfe-4762-841a-d222ac6f4aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:49.404980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-597814db-c120-4f99-9e6d-d7c88e891044 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:49.406130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-597814db-c120-4f99-9e6d-d7c88e891044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:49.412198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1299/2583] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:49.414211 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1285/2584] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:49.416743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b46808-2cfe-4762-841a-d222ac6f4aa6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.418214 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597814db-c120-4f99-9e6d-d7c88e891044 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.427302 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b46808-2cfe-4762-841a-d222ac6f4aa6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.427948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597814db-c120-4f99-9e6d-d7c88e891044 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.432594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507140d-f313-4597-a4be-dd855e534150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.433079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e3caa-2d1e-4f7c-818d-748973149909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.433322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507140d-f313-4597-a4be-dd855e534150 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.433650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507140d-f313-4597-a4be-dd855e534150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.434075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e3caa-2d1e-4f7c-818d-748973149909 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.434249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507140d-f313-4597-a4be-dd855e534150 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.434873 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e3caa-2d1e-4f7c-818d-748973149909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.435015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507140d-f313-4597-a4be-dd855e534150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:34:49.435282 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e3caa-2d1e-4f7c-818d-748973149909 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.435409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507140d-f313-4597-a4be-dd855e534150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.436016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e3caa-2d1e-4f7c-818d-748973149909 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:34:49.436322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e3caa-2d1e-4f7c-818d-748973149909 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.436916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507140d-f313-4597-a4be-dd855e534150 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.439722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e3caa-2d1e-4f7c-818d-748973149909 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.457081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1286/2585] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:34:49.458202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1300/2586] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:34:49.462671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e3caa-2d1e-4f7c-818d-748973149909 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.462851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507140d-f313-4597-a4be-dd855e534150 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.475124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507140d-f313-4597-a4be-dd855e534150 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.478590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e3caa-2d1e-4f7c-818d-748973149909 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.481218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa042ab-eb06-4423-a8bd-e334a99ea0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.481827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa042ab-eb06-4423-a8bd-e334a99ea0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.482274 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa042ab-eb06-4423-a8bd-e334a99ea0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.482454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa042ab-eb06-4423-a8bd-e334a99ea0e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.483066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa042ab-eb06-4423-a8bd-e334a99ea0e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.484996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa042ab-eb06-4423-a8bd-e334a99ea0e1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.486552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03db32-9282-4fe1-9ea0-ba7ce5c89a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.487673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03db32-9282-4fe1-9ea0-ba7ce5c89a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.488129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03db32-9282-4fe1-9ea0-ba7ce5c89a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.489775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03db32-9282-4fe1-9ea0-ba7ce5c89a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.490656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03db32-9282-4fe1-9ea0-ba7ce5c89a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.492740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03db32-9282-4fe1-9ea0-ba7ce5c89a17 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.515076 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1301/2587] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:34:49 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:34:49.519126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa042ab-eb06-4423-a8bd-e334a99ea0e1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.523031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1287/2588] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:34:49 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:34:49.528188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03db32-9282-4fe1-9ea0-ba7ce5c89a17 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.542633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:49.596756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03db32-9282-4fe1-9ea0-ba7ce5c89a17 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.597284 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa042ab-eb06-4423-a8bd-e334a99ea0e1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.602859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4b6a7-b50d-43b3-93cf-282906a89e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.603133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da18618-2557-4dc1-b067-d4361a95b453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.603542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4b6a7-b50d-43b3-93cf-282906a89e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.603816 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4b6a7-b50d-43b3-93cf-282906a89e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.603945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da18618-2557-4dc1-b067-d4361a95b453 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.604086 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4b6a7-b50d-43b3-93cf-282906a89e75 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.604203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da18618-2557-4dc1-b067-d4361a95b453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.604376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da18618-2557-4dc1-b067-d4361a95b453 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.604777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4b6a7-b50d-43b3-93cf-282906a89e75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.605066 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da18618-2557-4dc1-b067-d4361a95b453 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.607053 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da18618-2557-4dc1-b067-d4361a95b453 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.607181 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4b6a7-b50d-43b3-93cf-282906a89e75 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.639176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1302/2589] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:34:49 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:34:49.639406 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1288/2590] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:34:49 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:34:49.639553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4b6a7-b50d-43b3-93cf-282906a89e75 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.640217 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da18618-2557-4dc1-b067-d4361a95b453 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.649591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4b6a7-b50d-43b3-93cf-282906a89e75 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.650839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da18618-2557-4dc1-b067-d4361a95b453 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.655634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe9c9c-f6cc-4ebf-b169-159418ca001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.656588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe9c9c-f6cc-4ebf-b169-159418ca001a admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.656865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe9c9c-f6cc-4ebf-b169-159418ca001a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.657117 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3e69f-d937-4e01-a2f3-2eb756becde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.657281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe9c9c-f6cc-4ebf-b169-159418ca001a admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.657816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3e69f-d937-4e01-a2f3-2eb756becde1 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.657960 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe9c9c-f6cc-4ebf-b169-159418ca001a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.658087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3e69f-d937-4e01-a2f3-2eb756becde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.658356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3e69f-d937-4e01-a2f3-2eb756becde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.659132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3e69f-d937-4e01-a2f3-2eb756becde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.659486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe9c9c-f6cc-4ebf-b169-159418ca001a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.660730 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3e69f-d937-4e01-a2f3-2eb756becde1 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.670468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1303/2591] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 10:34:49.673815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1289/2592] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:34:49.674806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe9c9c-f6cc-4ebf-b169-159418ca001a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.675098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3e69f-d937-4e01-a2f3-2eb756becde1 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.685354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3e69f-d937-4e01-a2f3-2eb756becde1 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.685732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe9c9c-f6cc-4ebf-b169-159418ca001a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.691176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788fcb6-3a2c-44a7-827b-435559de11ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.691874 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788fcb6-3a2c-44a7-827b-435559de11ea admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.692019 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618218e-d622-434e-b8a0-a3fcc2be0eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.692154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788fcb6-3a2c-44a7-827b-435559de11ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.692442 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788fcb6-3a2c-44a7-827b-435559de11ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.692742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618218e-d622-434e-b8a0-a3fcc2be0eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.692947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618218e-d622-434e-b8a0-a3fcc2be0eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.693183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788fcb6-3a2c-44a7-827b-435559de11ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.693310 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618218e-d622-434e-b8a0-a3fcc2be0eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.693966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618218e-d622-434e-b8a0-a3fcc2be0eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.694858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788fcb6-3a2c-44a7-827b-435559de11ea admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.695874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618218e-d622-434e-b8a0-a3fcc2be0eb7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.708979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1290/2593] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 10:34:49.710717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1304/2594] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 10:34:49.713666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618218e-d622-434e-b8a0-a3fcc2be0eb7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.714271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788fcb6-3a2c-44a7-827b-435559de11ea admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.727449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618218e-d622-434e-b8a0-a3fcc2be0eb7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.727669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788fcb6-3a2c-44a7-827b-435559de11ea admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.733107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a2299-ab07-44a4-a105-167d679022aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.733561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafc643-9932-4ad1-9967-814f0eace247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.734261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a2299-ab07-44a4-a105-167d679022aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.734545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a2299-ab07-44a4-a105-167d679022aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.734736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc643-9932-4ad1-9967-814f0eace247 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.734843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a2299-ab07-44a4-a105-167d679022aa admin admin] PATH_INFO: `/v3/projects/63e5d1b42c9041bdb90705f0aa089371/users/39654338eea84cd18befa46f958735cb/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.734995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc643-9932-4ad1-9967-814f0eace247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.735358 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc643-9932-4ad1-9967-814f0eace247 admin admin] PATH_INFO: `/v3/projects/b73bddd1e4434ae6980546b225f2b341/users/dd9cb5379e6148c9b283fb26fc578c75/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.748021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:49.752930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a2299-ab07-44a4-a105-167d679022aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e5d1b42c9041bdb90705f0aa089371, user_id=39654338eea84cd18befa46f958735cb, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.755677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafc643-9932-4ad1-9967-814f0eace247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73bddd1e4434ae6980546b225f2b341, user_id=dd9cb5379e6148c9b283fb26fc578c75, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.756634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a2299-ab07-44a4-a105-167d679022aa admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.759202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafc643-9932-4ad1-9967-814f0eace247 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.775823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1291/2595] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:34:49 2022] PUT /identity/v3/projects/63e5d1b42c9041bdb90705f0aa089371/users/39654338eea84cd18befa46f958735cb/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:34:49.777828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1305/2596] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:34:49 2022] PUT /identity/v3/projects/b73bddd1e4434ae6980546b225f2b341/users/dd9cb5379e6148c9b283fb26fc578c75/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:34:49.783014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068524b-74ae-4dab-a1a5-00e51ebe039e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.783305 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068524b-74ae-4dab-a1a5-00e51ebe039e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.783585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068524b-74ae-4dab-a1a5-00e51ebe039e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.791172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356a4e-4077-411e-bc0a-4da04d8a239f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.791778 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356a4e-4077-411e-bc0a-4da04d8a239f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.792203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356a4e-4077-411e-bc0a-4da04d8a239f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.801718 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-9068524b-74ae-4dab-a1a5-00e51ebe039e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:49.802983 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9068524b-74ae-4dab-a1a5-00e51ebe039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:49.825328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ec356a4e-4077-411e-bc0a-4da04d8a239f None None] MFA Rules not processed for user `39654338eea84cd18befa46f958735cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:49.829031 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ec356a4e-4077-411e-bc0a-4da04d8a239f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:49.873200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1292/2597] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:49.887934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068524b-74ae-4dab-a1a5-00e51ebe039e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.889421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9068524b-74ae-4dab-a1a5-00e51ebe039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:49.914645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1306/2598] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:34:49.926189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec356a4e-4077-411e-bc0a-4da04d8a239f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:49.927676 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ec356a4e-4077-411e-bc0a-4da04d8a239f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:49.948871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068524b-74ae-4dab-a1a5-00e51ebe039e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:49.952744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f52f6c-08ad-4812-87e1-9371bcb1166c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:49.953400 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f52f6c-08ad-4812-87e1-9371bcb1166c service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:49.953680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f52f6c-08ad-4812-87e1-9371bcb1166c service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:49.953956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f52f6c-08ad-4812-87e1-9371bcb1166c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:49.963576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f52f6c-08ad-4812-87e1-9371bcb1166c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:49.965062 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f52f6c-08ad-4812-87e1-9371bcb1166c service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:49.975892 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1293/2599] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:49.998781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:50.006037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec356a4e-4077-411e-bc0a-4da04d8a239f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:50.012150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5647d-c3c4-4979-a4c0-e336975f4859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:50.012667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5647d-c3c4-4979-a4c0-e336975f4859 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:50.012778 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5647d-c3c4-4979-a4c0-e336975f4859 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:50.013193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5647d-c3c4-4979-a4c0-e336975f4859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:50.032929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5647d-c3c4-4979-a4c0-e336975f4859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:50.034538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5647d-c3c4-4979-a4c0-e336975f4859 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:50.047491 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1307/2600] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:34:54.387723 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf5683-cee2-42f6-9d19-809eaec9ab7b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:54.388244 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf5683-cee2-42f6-9d19-809eaec9ab7b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:54.388244 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf5683-cee2-42f6-9d19-809eaec9ab7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:54.428744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b4cf5683-cee2-42f6-9d19-809eaec9ab7b None None] MFA Rules not processed for user `dd9cb5379e6148c9b283fb26fc578c75`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:54.429951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b4cf5683-cee2-42f6-9d19-809eaec9ab7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:54.514030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5647d-c3c4-4979-a4c0-e336975f4859 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:54.519794 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1294/2601] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:34:54.544522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5647d-c3c4-4979-a4c0-e336975f4859 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:54.549306 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813e8c-2168-4f72-8a5c-379888aef351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:54.549977 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813e8c-2168-4f72-8a5c-379888aef351 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:54.550257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813e8c-2168-4f72-8a5c-379888aef351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:54.550528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813e8c-2168-4f72-8a5c-379888aef351 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:54.551256 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b813e8c-2168-4f72-8a5c-379888aef351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:54.553178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b813e8c-2168-4f72-8a5c-379888aef351 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:54.586268 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1308/2602] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:34:54 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:34:54.590685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf5683-cee2-42f6-9d19-809eaec9ab7b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:54.603581 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf5683-cee2-42f6-9d19-809eaec9ab7b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:54.626133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:34:54.670130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94772930-ab84-4247-a8cb-6fce44e720d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:54.670985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94772930-ab84-4247-a8cb-6fce44e720d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:54.671412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94772930-ab84-4247-a8cb-6fce44e720d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:54.671692 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94772930-ab84-4247-a8cb-6fce44e720d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:54.672531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94772930-ab84-4247-a8cb-6fce44e720d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:54.674492 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94772930-ab84-4247-a8cb-6fce44e720d8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:54.697861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1295/2603] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:34:54 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:34:54.702144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813e8c-2168-4f72-8a5c-379888aef351 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:54.712995 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813e8c-2168-4f72-8a5c-379888aef351 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:54.717842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e12ae-f1cb-4eba-82c4-91f40a1ae5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:54.718527 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e12ae-f1cb-4eba-82c4-91f40a1ae5af admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:54.718860 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e12ae-f1cb-4eba-82c4-91f40a1ae5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:54.719137 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e12ae-f1cb-4eba-82c4-91f40a1ae5af admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:54.719874 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e12ae-f1cb-4eba-82c4-91f40a1ae5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:54.721516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e12ae-f1cb-4eba-82c4-91f40a1ae5af admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:54.733615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1309/2604] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 10:34:54.737845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94772930-ab84-4247-a8cb-6fce44e720d8 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:54.748466 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94772930-ab84-4247-a8cb-6fce44e720d8 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:54.753259 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1f2e2-c033-42c0-b483-cbeeebc3c873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:54.754176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1f2e2-c033-42c0-b483-cbeeebc3c873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:54.754485 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1f2e2-c033-42c0-b483-cbeeebc3c873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:54.754795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1f2e2-c033-42c0-b483-cbeeebc3c873 admin admin] PATH_INFO: `/v3/projects/2e8464a3ee6646f88c02e93fca5a69bd/users/edab531bbacd47fd92cf49bef7b8a740/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:54.772795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1f2e2-c033-42c0-b483-cbeeebc3c873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e8464a3ee6646f88c02e93fca5a69bd, user_id=edab531bbacd47fd92cf49bef7b8a740, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:54.776222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1f2e2-c033-42c0-b483-cbeeebc3c873 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:54.792438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1296/2605] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:34:54 2022] PUT /identity/v3/projects/2e8464a3ee6646f88c02e93fca5a69bd/users/edab531bbacd47fd92cf49bef7b8a740/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:54.797220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e12ae-f1cb-4eba-82c4-91f40a1ae5af admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:54.810652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e12ae-f1cb-4eba-82c4-91f40a1ae5af admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:54.875048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c246d7-9c80-403b-ad15-3e9f926469ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:54.875766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c246d7-9c80-403b-ad15-3e9f926469ee admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:54.876107 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c246d7-9c80-403b-ad15-3e9f926469ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:54.876557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c246d7-9c80-403b-ad15-3e9f926469ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:54.877301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c246d7-9c80-403b-ad15-3e9f926469ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:54.879008 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c246d7-9c80-403b-ad15-3e9f926469ee admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:54.891136 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1310/2606] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:34:54.895324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1f2e2-c033-42c0-b483-cbeeebc3c873 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:54.908010 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1f2e2-c033-42c0-b483-cbeeebc3c873 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:54.913890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9dea1-ba2c-4285-adf7-d374605e3b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:54.914872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9dea1-ba2c-4285-adf7-d374605e3b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:54.915175 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9dea1-ba2c-4285-adf7-d374605e3b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:54.915473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9dea1-ba2c-4285-adf7-d374605e3b34 admin admin] PATH_INFO: `/v3/projects/2e8464a3ee6646f88c02e93fca5a69bd/users/edab531bbacd47fd92cf49bef7b8a740/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:54.917672 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9dea1-ba2c-4285-adf7-d374605e3b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e8464a3ee6646f88c02e93fca5a69bd, user_id=edab531bbacd47fd92cf49bef7b8a740, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:54.921139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9dea1-ba2c-4285-adf7-d374605e3b34 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:54.935594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1297/2607] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:34:54 2022] PUT /identity/v3/projects/2e8464a3ee6646f88c02e93fca5a69bd/users/edab531bbacd47fd92cf49bef7b8a740/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:34:59.228223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c246d7-9c80-403b-ad15-3e9f926469ee admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:34:59.232764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-01c246d7-9c80-403b-ad15-3e9f926469ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:59.352486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c246d7-9c80-403b-ad15-3e9f926469ee admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:34:59.446261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c8676-f507-4485-9621-7e2a81998761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:34:59.447016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c8676-f507-4485-9621-7e2a81998761 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:59.447325 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c8676-f507-4485-9621-7e2a81998761 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:59.447778 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c8676-f507-4485-9621-7e2a81998761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:59.454336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ae9c8676-f507-4485-9621-7e2a81998761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:34:59.476875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c8676-f507-4485-9621-7e2a81998761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:34:59.480593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c8676-f507-4485-9621-7e2a81998761 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:34:59.512101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1311/2608] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:34:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:34:59.881336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc003-5ec6-4020-89a1-8e896009aaf9 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:34:59.882059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc003-5ec6-4020-89a1-8e896009aaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:34:59.882059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc003-5ec6-4020-89a1-8e896009aaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:34:59.934808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-311cc003-5ec6-4020-89a1-8e896009aaf9 None None] MFA Rules not processed for user `edab531bbacd47fd92cf49bef7b8a740`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:34:59.936911 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-311cc003-5ec6-4020-89a1-8e896009aaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:00.068890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1298/2609] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:34:59 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:00.100736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c8676-f507-4485-9621-7e2a81998761 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:00.103048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ae9c8676-f507-4485-9621-7e2a81998761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:00.130295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:00.233131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c8676-f507-4485-9621-7e2a81998761 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:00.240128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca4acc-cae9-44a6-b4c8-5dd3c4b7bf79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:00.240844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4acc-cae9-44a6-b4c8-5dd3c4b7bf79 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:00.241131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4acc-cae9-44a6-b4c8-5dd3c4b7bf79 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:00.241546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4acc-cae9-44a6-b4c8-5dd3c4b7bf79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:00.327674 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca4acc-cae9-44a6-b4c8-5dd3c4b7bf79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:00.329545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca4acc-cae9-44a6-b4c8-5dd3c4b7bf79 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:00.341682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1312/2610] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:35:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:35:01.480447 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9692452-5d1c-44b4-9d8f-27e3c9c932ce None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:01.481365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9692452-5d1c-44b4-9d8f-27e3c9c932ce None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:01.481651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9692452-5d1c-44b4-9d8f-27e3c9c932ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:01.500374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a9692452-5d1c-44b4-9d8f-27e3c9c932ce None None] MFA Rules not processed for user `dd9cb5379e6148c9b283fb26fc578c75`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:35:01.501609 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a9692452-5d1c-44b4-9d8f-27e3c9c932ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:01.508648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1299/2611] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:35:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:07.971467 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca4acc-cae9-44a6-b4c8-5dd3c4b7bf79 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:08.054990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca4acc-cae9-44a6-b4c8-5dd3c4b7bf79 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:08.060465 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:08.061399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:08.061678 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:08.061991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] PATH_INFO: `/v3/users/39654338eea84cd18befa46f958735cb` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:08.063264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39654338eea84cd18befa46f958735cb)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:08.065293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:08.132960 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39654338eea84cd18befa46f958735cb'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:08.139512 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:08.151665 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39654338eea84cd18befa46f958735cb'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:08.152729 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39654338eea84cd18befa46f958735cb'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:08.162556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39654338eea84cd18befa46f958735cb'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:08.163689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39654338eea84cd18befa46f958735cb'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:08.184013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1313/2612] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:35:07 2022] DELETE /identity/v3/users/39654338eea84cd18befa46f958735cb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:08.189273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4056146-ebb7-4b1c-b0da-bf0b1deff39f None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:08.189673 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4056146-ebb7-4b1c-b0da-bf0b1deff39f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:08.189754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4056146-ebb7-4b1c-b0da-bf0b1deff39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:08.223431 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-b4056146-ebb7-4b1c-b0da-bf0b1deff39f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:35:08.224724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b4056146-ebb7-4b1c-b0da-bf0b1deff39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:08.285905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:08.304878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1300/2613] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:08.317735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:08.319787 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:08.395848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe5af2-0edc-4ab1-b26e-4c62c9b6d965 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:08.400164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4c9c4-2ded-4202-a503-f5192044a44f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:08.400653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4c9c4-2ded-4202-a503-f5192044a44f service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:08.400927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4c9c4-2ded-4202-a503-f5192044a44f service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:08.401273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4c9c4-2ded-4202-a503-f5192044a44f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:08.414598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4c9c4-2ded-4202-a503-f5192044a44f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:08.416083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4c9c4-2ded-4202-a503-f5192044a44f service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:08.430930 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1314/2614] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:35:08.744568 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4056146-ebb7-4b1c-b0da-bf0b1deff39f None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:08.746110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b4056146-ebb7-4b1c-b0da-bf0b1deff39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:08.759744 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4056146-ebb7-4b1c-b0da-bf0b1deff39f None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:08.765682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:08.766699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:08.767027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:08.767370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] PATH_INFO: `/v3/projects/63e5d1b42c9041bdb90705f0aa089371` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:08.769015 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e5d1b42c9041bdb90705f0aa089371)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:08.771164 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:08.831579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e5d1b42c9041bdb90705f0aa089371', 'request_id': 'req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:08.841035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e5d1b42c9041bdb90705f0aa089371', 'request_id': 'req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:08.848112 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e5d1b42c9041bdb90705f0aa089371', 'request_id': 'req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:08.860521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] The token cache is being invalidate because project 63e5d1b42c9041bdb90705f0aa089371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:35:08.860792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:08.861552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:08.863532 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1301/2615] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:35:08 2022] DELETE /identity/v3/projects/63e5d1b42c9041bdb90705f0aa089371 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:17.977386 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4c9c4-2ded-4202-a503-f5192044a44f service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:17.979279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d9f4c9c4-2ded-4202-a503-f5192044a44f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:18.041138 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4c9c4-2ded-4202-a503-f5192044a44f service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:18.091201 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef90ba7-0a7a-4bec-86a2-bd3a54fbe941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:18.091843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef90ba7-0a7a-4bec-86a2-bd3a54fbe941 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:18.092119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef90ba7-0a7a-4bec-86a2-bd3a54fbe941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:18.092392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef90ba7-0a7a-4bec-86a2-bd3a54fbe941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:18.094434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7ef90ba7-0a7a-4bec-86a2-bd3a54fbe941 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:18.139090 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:18.170394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef90ba7-0a7a-4bec-86a2-bd3a54fbe941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:18.171897 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef90ba7-0a7a-4bec-86a2-bd3a54fbe941 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:18.183652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1315/2616] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:35:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:35:18.501502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:18.503130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:18.574987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa84fe9-e9e2-4cc5-802f-a1e4c77c33cf admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:18.579018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430aedc9-6e2e-4e0b-8760-ceadd3a5ef91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:18.579686 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430aedc9-6e2e-4e0b-8760-ceadd3a5ef91 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:18.579966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430aedc9-6e2e-4e0b-8760-ceadd3a5ef91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:18.580242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430aedc9-6e2e-4e0b-8760-ceadd3a5ef91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:18.582316 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-430aedc9-6e2e-4e0b-8760-ceadd3a5ef91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:18.592742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430aedc9-6e2e-4e0b-8760-ceadd3a5ef91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:18.594191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430aedc9-6e2e-4e0b-8760-ceadd3a5ef91 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:18.602665 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1302/2617] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:35:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:35:19.104959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef90ba7-0a7a-4bec-86a2-bd3a54fbe941 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:19.106646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7ef90ba7-0a7a-4bec-86a2-bd3a54fbe941 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:19.119229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef90ba7-0a7a-4bec-86a2-bd3a54fbe941 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:19.123636 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14558039-52ab-4908-865a-75cb5cb9e5dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:19.124269 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14558039-52ab-4908-865a-75cb5cb9e5dc service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:19.124722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14558039-52ab-4908-865a-75cb5cb9e5dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:19.125003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14558039-52ab-4908-865a-75cb5cb9e5dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:19.127139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-14558039-52ab-4908-865a-75cb5cb9e5dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:19.142494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14558039-52ab-4908-865a-75cb5cb9e5dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:19.143957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14558039-52ab-4908-865a-75cb5cb9e5dc service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:19.155481 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1316/2618] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:35:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:35:28.598903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:29.150364 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=20 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:38.634872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430aedc9-6e2e-4e0b-8760-ceadd3a5ef91 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:38.643982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430aedc9-6e2e-4e0b-8760-ceadd3a5ef91 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:38.647782 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef88ffd-bbe9-4507-9e26-a259377093d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:38.648442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef88ffd-bbe9-4507-9e26-a259377093d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:38.648719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef88ffd-bbe9-4507-9e26-a259377093d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:38.648992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef88ffd-bbe9-4507-9e26-a259377093d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:38.650939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6ef88ffd-bbe9-4507-9e26-a259377093d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:38.726599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef88ffd-bbe9-4507-9e26-a259377093d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:38.728076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef88ffd-bbe9-4507-9e26-a259377093d8 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:38.740162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1303/2619] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:35:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:35:41.532661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14558039-52ab-4908-865a-75cb5cb9e5dc service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:41.543157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14558039-52ab-4908-865a-75cb5cb9e5dc service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:41.547229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d270516-ae23-4d06-b9a1-15952705909c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:41.547904 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d270516-ae23-4d06-b9a1-15952705909c service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:41.548192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d270516-ae23-4d06-b9a1-15952705909c service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:41.548471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d270516-ae23-4d06-b9a1-15952705909c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:41.550328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7d270516-ae23-4d06-b9a1-15952705909c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:41.630323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d270516-ae23-4d06-b9a1-15952705909c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:41.631880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d270516-ae23-4d06-b9a1-15952705909c service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:41.643513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1317/2620] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:35:45.058193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef88ffd-bbe9-4507-9e26-a259377093d8 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:45.059845 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6ef88ffd-bbe9-4507-9e26-a259377093d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:45.083737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef88ffd-bbe9-4507-9e26-a259377093d8 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:45.088921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:45.089756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:45.090036 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:45.090376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] PATH_INFO: `/v3/users/229771b215b3492abc2c5ec38ff0d929` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:45.106213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] RBAC: Authorizing `identity:delete_user(user_id=229771b215b3492abc2c5ec38ff0d929)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:45.108243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:45.162946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229771b215b3492abc2c5ec38ff0d929'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:45.176642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229771b215b3492abc2c5ec38ff0d929'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:45.177455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229771b215b3492abc2c5ec38ff0d929'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:45.178148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229771b215b3492abc2c5ec38ff0d929'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:45.186117 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229771b215b3492abc2c5ec38ff0d929'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:45.203057 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1304/2621] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:35:45 2022] DELETE /identity/v3/users/229771b215b3492abc2c5ec38ff0d929 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:45.210750 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d270516-ae23-4d06-b9a1-15952705909c service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:45.212388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7d270516-ae23-4d06-b9a1-15952705909c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:45.270894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d270516-ae23-4d06-b9a1-15952705909c service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:45.336528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac509f-8f87-4590-a785-0cd4610627b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:45.337191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac509f-8f87-4590-a785-0cd4610627b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:45.337462 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac509f-8f87-4590-a785-0cd4610627b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:45.337836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac509f-8f87-4590-a785-0cd4610627b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:45.339954 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f0ac509f-8f87-4590-a785-0cd4610627b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:45.427088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:45.429227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:45.446017 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac509f-8f87-4590-a785-0cd4610627b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:45.447798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac509f-8f87-4590-a785-0cd4610627b8 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:45.463289 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1318/2622] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:35:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:35:45.511158 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:45.519793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca19bc-e6c4-4f3b-b17a-9e483c3cdaee admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:45.526870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:45.527836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:45.528116 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:45.528390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] PATH_INFO: `/v3/projects/702000a5942b41689821164e169520d8` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:45.530006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=702000a5942b41689821164e169520d8)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:45.532487 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:45.591001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '702000a5942b41689821164e169520d8', 'request_id': 'req-d47c649e-c177-4cf3-b601-2f743b73e5e1'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:45.601136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '702000a5942b41689821164e169520d8', 'request_id': 'req-d47c649e-c177-4cf3-b601-2f743b73e5e1'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:45.611695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '702000a5942b41689821164e169520d8', 'request_id': 'req-d47c649e-c177-4cf3-b601-2f743b73e5e1'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:45.623985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] The token cache is being invalidate because project 702000a5942b41689821164e169520d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:35:45.624355 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:45.625206 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d47c649e-c177-4cf3-b601-2f743b73e5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:45.626902 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1305/2623] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:35:45 2022] DELETE /identity/v3/projects/702000a5942b41689821164e169520d8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:45.633580 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f858f51-09cc-4968-b7af-169e16740f3c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:45.633862 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f858f51-09cc-4968-b7af-169e16740f3c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:45.634126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f858f51-09cc-4968-b7af-169e16740f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:45.664336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2f858f51-09cc-4968-b7af-169e16740f3c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:35:45.665575 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2f858f51-09cc-4968-b7af-169e16740f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:45.680378 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:45.734701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1319/2624] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:45.743521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b18c2-6c34-4cef-b050-1acd8fc6c348 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:45.743775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b18c2-6c34-4cef-b050-1acd8fc6c348 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:45.744095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b18c2-6c34-4cef-b050-1acd8fc6c348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:45.770254 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-765b18c2-6c34-4cef-b050-1acd8fc6c348 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:35:45.771547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-765b18c2-6c34-4cef-b050-1acd8fc6c348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:45.781142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1306/2625] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:45.786056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f858f51-09cc-4968-b7af-169e16740f3c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:45.796620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f858f51-09cc-4968-b7af-169e16740f3c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:45.801733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1f8ca-0c78-447a-8c3e-2948062a8e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:45.802417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1f8ca-0c78-447a-8c3e-2948062a8e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:45.802860 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1f8ca-0c78-447a-8c3e-2948062a8e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:45.803134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1f8ca-0c78-447a-8c3e-2948062a8e0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:45.803873 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1f8ca-0c78-447a-8c3e-2948062a8e0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:35:45.804176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1f8ca-0c78-447a-8c3e-2948062a8e0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:45.805824 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1f8ca-0c78-447a-8c3e-2948062a8e0d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:45.819620 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1320/2626] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:35:45.824525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b18c2-6c34-4cef-b050-1acd8fc6c348 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:45.836306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b18c2-6c34-4cef-b050-1acd8fc6c348 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:45.841851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e44b8-b294-4238-9a37-91568e01776c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:45.842530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e44b8-b294-4238-9a37-91568e01776c admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:45.843772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e44b8-b294-4238-9a37-91568e01776c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:45.844060 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e44b8-b294-4238-9a37-91568e01776c admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:45.844764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e44b8-b294-4238-9a37-91568e01776c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:45.846727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e44b8-b294-4238-9a37-91568e01776c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:45.877907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1307/2627] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:35:45 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:35:45.882430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1f8ca-0c78-447a-8c3e-2948062a8e0d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:45.893922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1f8ca-0c78-447a-8c3e-2948062a8e0d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:45.968299 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc1eae-f3d6-490c-8e35-cde542d92a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:45.968999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc1eae-f3d6-490c-8e35-cde542d92a51 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:45.969281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc1eae-f3d6-490c-8e35-cde542d92a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:45.969559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc1eae-f3d6-490c-8e35-cde542d92a51 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:45.970316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc1eae-f3d6-490c-8e35-cde542d92a51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:45.972430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc1eae-f3d6-490c-8e35-cde542d92a51 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:45.997578 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1321/2628] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:35:45 2022] POST /identity/v3/users => generated 303 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:35:46.006871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e44b8-b294-4238-9a37-91568e01776c admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:46.018147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e44b8-b294-4238-9a37-91568e01776c admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:46.023108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3843a-66f8-487c-945b-fbe1889d5004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:46.023789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3843a-66f8-487c-945b-fbe1889d5004 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:46.024063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3843a-66f8-487c-945b-fbe1889d5004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:46.024334 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3843a-66f8-487c-945b-fbe1889d5004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:46.025065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d3843a-66f8-487c-945b-fbe1889d5004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:46.026700 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d3843a-66f8-487c-945b-fbe1889d5004 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:46.043710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1308/2629] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:35:46.044146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc1eae-f3d6-490c-8e35-cde542d92a51 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:46.055553 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc1eae-f3d6-490c-8e35-cde542d92a51 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:46.061265 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98918ec-c677-4fc1-a5b5-6279ed238aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:46.062178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98918ec-c677-4fc1-a5b5-6279ed238aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:46.062395 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98918ec-c677-4fc1-a5b5-6279ed238aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:46.062637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98918ec-c677-4fc1-a5b5-6279ed238aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:46.063374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98918ec-c677-4fc1-a5b5-6279ed238aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:46.065050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98918ec-c677-4fc1-a5b5-6279ed238aa4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:46.078366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1322/2630] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:35:46.083927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3843a-66f8-487c-945b-fbe1889d5004 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:46.105133 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3843a-66f8-487c-945b-fbe1889d5004 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:46.110122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb720a-2028-4891-be41-c727dbb97bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:46.111067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb720a-2028-4891-be41-c727dbb97bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:46.111353 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb720a-2028-4891-be41-c727dbb97bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:46.111617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb720a-2028-4891-be41-c727dbb97bfe admin admin] PATH_INFO: `/v3/projects/62a062d1d9e0481794eb3c7af2b1a4ec/users/b6622fd8f95d4b01923d017f6013f460/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:46.129170 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb720a-2028-4891-be41-c727dbb97bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a062d1d9e0481794eb3c7af2b1a4ec, user_id=b6622fd8f95d4b01923d017f6013f460, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:46.131931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb720a-2028-4891-be41-c727dbb97bfe admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:46.150224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1309/2631] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:35:46 2022] PUT /identity/v3/projects/62a062d1d9e0481794eb3c7af2b1a4ec/users/b6622fd8f95d4b01923d017f6013f460/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:35:46.156446 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5d209-ee61-4095-9deb-ec10e3a14dc0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:46.156724 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5d209-ee61-4095-9deb-ec10e3a14dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:46.156996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5d209-ee61-4095-9deb-ec10e3a14dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:46.181006 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1aa5d209-ee61-4095-9deb-ec10e3a14dc0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:35:46.182677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1aa5d209-ee61-4095-9deb-ec10e3a14dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:46.269694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1323/2632] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:46.281408 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb720a-2028-4891-be41-c727dbb97bfe admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:46.283856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-0beb720a-2028-4891-be41-c727dbb97bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:46.378722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb720a-2028-4891-be41-c727dbb97bfe admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:46.383660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df045105-0e95-4621-a8f6-16a265732aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:46.384344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df045105-0e95-4621-a8f6-16a265732aed service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:46.384649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df045105-0e95-4621-a8f6-16a265732aed service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:46.384951 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df045105-0e95-4621-a8f6-16a265732aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:46.396916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df045105-0e95-4621-a8f6-16a265732aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:46.398787 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df045105-0e95-4621-a8f6-16a265732aed service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:46.410174 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1310/2633] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:35:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:35:47.152192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08d2dd-c21a-4dfe-8823-ef1fe689477d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:47.152800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08d2dd-c21a-4dfe-8823-ef1fe689477d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:47.153076 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08d2dd-c21a-4dfe-8823-ef1fe689477d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:47.190964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2a08d2dd-c21a-4dfe-8823-ef1fe689477d None None] MFA Rules not processed for user `b6622fd8f95d4b01923d017f6013f460`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:35:47.192262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2a08d2dd-c21a-4dfe-8823-ef1fe689477d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:47.211867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:47.303319 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1324/2634] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:35:47 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:47.310258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df045105-0e95-4621-a8f6-16a265732aed service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:47.327671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df045105-0e95-4621-a8f6-16a265732aed service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:47.333438 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e466f-fa4f-4f83-96ce-1c594f8fe670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:47.334120 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e466f-fa4f-4f83-96ce-1c594f8fe670 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:47.334410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e466f-fa4f-4f83-96ce-1c594f8fe670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:47.334712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e466f-fa4f-4f83-96ce-1c594f8fe670 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:47.335428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e466f-fa4f-4f83-96ce-1c594f8fe670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:47.337399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e466f-fa4f-4f83-96ce-1c594f8fe670 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:47.376858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1311/2635] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:35:47 2022] POST /identity/v3/projects => generated 403 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:35:47.382409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08d2dd-c21a-4dfe-8823-ef1fe689477d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:47.396551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08d2dd-c21a-4dfe-8823-ef1fe689477d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:47.468792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdc276-eee5-463b-a0c1-8ea3d25c67bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:47.469688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdc276-eee5-463b-a0c1-8ea3d25c67bc admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:47.470010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdc276-eee5-463b-a0c1-8ea3d25c67bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:47.470324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdc276-eee5-463b-a0c1-8ea3d25c67bc admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:47.471170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcdc276-eee5-463b-a0c1-8ea3d25c67bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:47.473287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcdc276-eee5-463b-a0c1-8ea3d25c67bc admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:47.498408 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1325/2636] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:35:47 2022] POST /identity/v3/users => generated 308 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:35:47.505769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e466f-fa4f-4f83-96ce-1c594f8fe670 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:47.512107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:47.522054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e466f-fa4f-4f83-96ce-1c594f8fe670 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:47.529525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713db75-a9f5-4be5-9b80-7851afeb0848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:47.530394 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713db75-a9f5-4be5-9b80-7851afeb0848 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:47.530706 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713db75-a9f5-4be5-9b80-7851afeb0848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:47.530986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713db75-a9f5-4be5-9b80-7851afeb0848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:47.531731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713db75-a9f5-4be5-9b80-7851afeb0848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:47.533359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713db75-a9f5-4be5-9b80-7851afeb0848 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:47.545158 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1312/2637] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:35:47.551402 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdc276-eee5-463b-a0c1-8ea3d25c67bc admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:47.564758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdc276-eee5-463b-a0c1-8ea3d25c67bc admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:47.570530 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5efc0-d604-41f9-9cb9-dbc14c988ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:47.571475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5efc0-d604-41f9-9cb9-dbc14c988ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:47.571758 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5efc0-d604-41f9-9cb9-dbc14c988ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:47.572035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5efc0-d604-41f9-9cb9-dbc14c988ea8 admin admin] PATH_INFO: `/v3/projects/3db47068b0ea4a5da53f2d3aeb8b5daf/users/02a13760676645ef9d1df4a637ab5fa5/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:47.590211 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5efc0-d604-41f9-9cb9-dbc14c988ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3db47068b0ea4a5da53f2d3aeb8b5daf, user_id=02a13760676645ef9d1df4a637ab5fa5, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:47.593111 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5efc0-d604-41f9-9cb9-dbc14c988ea8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:47.612223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1326/2638] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:35:47 2022] PUT /identity/v3/projects/3db47068b0ea4a5da53f2d3aeb8b5daf/users/02a13760676645ef9d1df4a637ab5fa5/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:47.616594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713db75-a9f5-4be5-9b80-7851afeb0848 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:47.628884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713db75-a9f5-4be5-9b80-7851afeb0848 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:47.695129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c630c-c5f8-4e76-8239-089799fcf774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:47.695803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c630c-c5f8-4e76-8239-089799fcf774 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:47.696075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c630c-c5f8-4e76-8239-089799fcf774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:47.696346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c630c-c5f8-4e76-8239-089799fcf774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:47.697068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c630c-c5f8-4e76-8239-089799fcf774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:47.698703 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c630c-c5f8-4e76-8239-089799fcf774 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:47.710250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1313/2639] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:35:47.715118 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5efc0-d604-41f9-9cb9-dbc14c988ea8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:47.728621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5efc0-d604-41f9-9cb9-dbc14c988ea8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:47.735812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60848e-9625-4fbe-8d07-2bd72fac4217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:47.736734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60848e-9625-4fbe-8d07-2bd72fac4217 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:47.737299 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60848e-9625-4fbe-8d07-2bd72fac4217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:47.737590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60848e-9625-4fbe-8d07-2bd72fac4217 admin admin] PATH_INFO: `/v3/projects/3db47068b0ea4a5da53f2d3aeb8b5daf/users/02a13760676645ef9d1df4a637ab5fa5/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:47.740103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60848e-9625-4fbe-8d07-2bd72fac4217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3db47068b0ea4a5da53f2d3aeb8b5daf, user_id=02a13760676645ef9d1df4a637ab5fa5, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:47.744249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60848e-9625-4fbe-8d07-2bd72fac4217 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:47.757424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1327/2640] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:35:47 2022] PUT /identity/v3/projects/3db47068b0ea4a5da53f2d3aeb8b5daf/users/02a13760676645ef9d1df4a637ab5fa5/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:48.814105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa518daf-d061-4d4a-8811-d22256c7dbab None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:48.814829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa518daf-d061-4d4a-8811-d22256c7dbab None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:48.814829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa518daf-d061-4d4a-8811-d22256c7dbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:48.855646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fa518daf-d061-4d4a-8811-d22256c7dbab None None] MFA Rules not processed for user `02a13760676645ef9d1df4a637ab5fa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:35:48.856922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fa518daf-d061-4d4a-8811-d22256c7dbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:48.938765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1314/2641] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:35:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:50.118324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60848e-9625-4fbe-8d07-2bd72fac4217 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:50.186850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60848e-9625-4fbe-8d07-2bd72fac4217 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:50.203052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:50.203987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:50.204263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:50.204535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] PATH_INFO: `/v3/users/b6622fd8f95d4b01923d017f6013f460` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:50.205886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6622fd8f95d4b01923d017f6013f460)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:50.207894 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:50.214614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:50.281936 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6622fd8f95d4b01923d017f6013f460'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.300473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6622fd8f95d4b01923d017f6013f460'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.301230 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6622fd8f95d4b01923d017f6013f460'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.308659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6622fd8f95d4b01923d017f6013f460'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.309454 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6622fd8f95d4b01923d017f6013f460'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.324010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1328/2642] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:35:50 2022] DELETE /identity/v3/users/b6622fd8f95d4b01923d017f6013f460 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:50.523572 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa518daf-d061-4d4a-8811-d22256c7dbab None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:50.536464 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa518daf-d061-4d4a-8811-d22256c7dbab None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:50.615337 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:50.616265 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:50.616710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:50.616859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] PATH_INFO: `/v3/projects/62a062d1d9e0481794eb3c7af2b1a4ec` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:50.618060 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a062d1d9e0481794eb3c7af2b1a4ec)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:50.620098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:50.637901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:50.685920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a062d1d9e0481794eb3c7af2b1a4ec', 'request_id': 'req-6dcef763-54d7-405a-9460-19944f044f98'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.687642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a062d1d9e0481794eb3c7af2b1a4ec', 'request_id': 'req-6dcef763-54d7-405a-9460-19944f044f98'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.696146 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a062d1d9e0481794eb3c7af2b1a4ec', 'request_id': 'req-6dcef763-54d7-405a-9460-19944f044f98'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.708421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] The token cache is being invalidate because project 62a062d1d9e0481794eb3c7af2b1a4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:35:50.708708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.709393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.711031 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1315/2643] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:35:50 2022] DELETE /identity/v3/projects/62a062d1d9e0481794eb3c7af2b1a4ec => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:50.716105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:50.718131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:50.732238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995db2db-e8c7-42b2-be91-03681708cd9d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:50.816180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15f006-a415-4006-8c0a-35140845fd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:50.817754 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15f006-a415-4006-8c0a-35140845fd2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:50.818151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15f006-a415-4006-8c0a-35140845fd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:50.818659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15f006-a415-4006-8c0a-35140845fd2a admin admin] PATH_INFO: `/v3/users/02a13760676645ef9d1df4a637ab5fa5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:50.820499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15f006-a415-4006-8c0a-35140845fd2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a13760676645ef9d1df4a637ab5fa5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:50.822598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15f006-a415-4006-8c0a-35140845fd2a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:50.878701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4e15f006-a415-4006-8c0a-35140845fd2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a13760676645ef9d1df4a637ab5fa5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.893158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4e15f006-a415-4006-8c0a-35140845fd2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a13760676645ef9d1df4a637ab5fa5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.893898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4e15f006-a415-4006-8c0a-35140845fd2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a13760676645ef9d1df4a637ab5fa5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.902019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4e15f006-a415-4006-8c0a-35140845fd2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a13760676645ef9d1df4a637ab5fa5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.902796 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4e15f006-a415-4006-8c0a-35140845fd2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a13760676645ef9d1df4a637ab5fa5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:50.917151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1329/2644] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:35:50 2022] DELETE /identity/v3/users/02a13760676645ef9d1df4a637ab5fa5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:51.115253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:51.116813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:51.131135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcef763-54d7-405a-9460-19944f044f98 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:51.199558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:51.200432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:51.200709 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:51.200974 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] PATH_INFO: `/v3/projects/3db47068b0ea4a5da53f2d3aeb8b5daf` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:51.202734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3db47068b0ea4a5da53f2d3aeb8b5daf)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:51.204839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:51.261345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3db47068b0ea4a5da53f2d3aeb8b5daf', 'request_id': 'req-da4cab38-a72f-42e2-bd5e-d88956172f56'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:51.270411 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3db47068b0ea4a5da53f2d3aeb8b5daf', 'request_id': 'req-da4cab38-a72f-42e2-bd5e-d88956172f56'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:51.277125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3db47068b0ea4a5da53f2d3aeb8b5daf', 'request_id': 'req-da4cab38-a72f-42e2-bd5e-d88956172f56'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:51.289726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] The token cache is being invalidate because project 3db47068b0ea4a5da53f2d3aeb8b5daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:35:51.290023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:51.290798 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-da4cab38-a72f-42e2-bd5e-d88956172f56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:35:51.292477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1316/2645] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:35:51 2022] DELETE /identity/v3/projects/3db47068b0ea4a5da53f2d3aeb8b5daf => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:35:51.297706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2dc83-82f6-43cf-880c-77327acc0a8c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:51.298232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2dc83-82f6-43cf-880c-77327acc0a8c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:51.298232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2dc83-82f6-43cf-880c-77327acc0a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:51.307262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:51.323132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5ac2dc83-82f6-43cf-880c-77327acc0a8c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:35:51.324546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5ac2dc83-82f6-43cf-880c-77327acc0a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:51.391083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1330/2646] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:51.400358 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f6624-4c66-452e-8781-d712d9670ad0 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:51.400539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f6624-4c66-452e-8781-d712d9670ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:51.400829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f6624-4c66-452e-8781-d712d9670ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:51.421271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-ee7f6624-4c66-452e-8781-d712d9670ad0 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:35:51.422664 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ee7f6624-4c66-452e-8781-d712d9670ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:51.431971 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1317/2647] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:51.437410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2dc83-82f6-43cf-880c-77327acc0a8c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:51.447568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2dc83-82f6-43cf-880c-77327acc0a8c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:51.453525 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82746d30-7287-4ed4-9726-2dc582be6c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:51.454196 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82746d30-7287-4ed4-9726-2dc582be6c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:51.454476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82746d30-7287-4ed4-9726-2dc582be6c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:51.454785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82746d30-7287-4ed4-9726-2dc582be6c86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:51.455523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82746d30-7287-4ed4-9726-2dc582be6c86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:35:51.455837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82746d30-7287-4ed4-9726-2dc582be6c86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:51.457486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82746d30-7287-4ed4-9726-2dc582be6c86 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:51.469945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1331/2648] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:35:51.475522 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f6624-4c66-452e-8781-d712d9670ad0 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:51.485714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f6624-4c66-452e-8781-d712d9670ad0 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:51.491016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b7b8c-1832-4823-89f8-922ffc07240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:51.491682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b7b8c-1832-4823-89f8-922ffc07240c admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:51.491955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b7b8c-1832-4823-89f8-922ffc07240c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:51.492222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b7b8c-1832-4823-89f8-922ffc07240c admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:51.492941 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b7b8c-1832-4823-89f8-922ffc07240c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:51.494861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b7b8c-1832-4823-89f8-922ffc07240c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:51.525567 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1318/2649] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:35:51 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:35:51.530178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82746d30-7287-4ed4-9726-2dc582be6c86 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:51.543445 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82746d30-7287-4ed4-9726-2dc582be6c86 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:51.608433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fbf83-b413-498d-94ca-05cfb1c65c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:51.609114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fbf83-b413-498d-94ca-05cfb1c65c68 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:51.609424 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fbf83-b413-498d-94ca-05cfb1c65c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:51.609922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fbf83-b413-498d-94ca-05cfb1c65c68 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:51.610655 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fbf83-b413-498d-94ca-05cfb1c65c68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:51.612639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fbf83-b413-498d-94ca-05cfb1c65c68 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:51.645960 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1332/2650] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:35:51 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:35:51.652179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b7b8c-1832-4823-89f8-922ffc07240c admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:51.664105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:51.669361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b7b8c-1832-4823-89f8-922ffc07240c admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:51.678335 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67963433-4132-4c77-81a1-9bf11aee5ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:51.679492 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67963433-4132-4c77-81a1-9bf11aee5ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:51.679958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67963433-4132-4c77-81a1-9bf11aee5ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:51.680570 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67963433-4132-4c77-81a1-9bf11aee5ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:51.682188 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67963433-4132-4c77-81a1-9bf11aee5ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:51.685407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67963433-4132-4c77-81a1-9bf11aee5ce3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:51.698616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1319/2651] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:35:51.701996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fbf83-b413-498d-94ca-05cfb1c65c68 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:51.713486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fbf83-b413-498d-94ca-05cfb1c65c68 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:51.719726 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3db850-1ceb-4b15-af18-7617e565abbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:51.720433 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3db850-1ceb-4b15-af18-7617e565abbd admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:51.720783 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3db850-1ceb-4b15-af18-7617e565abbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:51.721193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3db850-1ceb-4b15-af18-7617e565abbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:51.722324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3db850-1ceb-4b15-af18-7617e565abbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:51.724958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3db850-1ceb-4b15-af18-7617e565abbd admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:51.739893 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1333/2652] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:35:51.744477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67963433-4132-4c77-81a1-9bf11aee5ce3 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:51.756979 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67963433-4132-4c77-81a1-9bf11aee5ce3 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:51.762052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86f5ce-f0e6-43b0-bb52-72b2292f2dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:51.762994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86f5ce-f0e6-43b0-bb52-72b2292f2dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:51.763267 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86f5ce-f0e6-43b0-bb52-72b2292f2dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:51.763567 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86f5ce-f0e6-43b0-bb52-72b2292f2dfd admin admin] PATH_INFO: `/v3/projects/f13e186b878a4c3fb356ad6ee449a134/users/176a4cd7bb0d42c4b495464e4f4369f3/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:51.779820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86f5ce-f0e6-43b0-bb52-72b2292f2dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13e186b878a4c3fb356ad6ee449a134, user_id=176a4cd7bb0d42c4b495464e4f4369f3, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:51.782533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86f5ce-f0e6-43b0-bb52-72b2292f2dfd admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:51.799859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1320/2653] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:35:51 2022] PUT /identity/v3/projects/f13e186b878a4c3fb356ad6ee449a134/users/176a4cd7bb0d42c4b495464e4f4369f3/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:51.810656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a50007-5685-473b-aef9-bb5404bce001 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:51.811032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a50007-5685-473b-aef9-bb5404bce001 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:51.811389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a50007-5685-473b-aef9-bb5404bce001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:51.841694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-92a50007-5685-473b-aef9-bb5404bce001 None None] MFA Rules not processed for user `176a4cd7bb0d42c4b495464e4f4369f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:35:51.842858 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-92a50007-5685-473b-aef9-bb5404bce001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:51.921945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:51.934827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1334/2654] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:35:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:51.940552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86f5ce-f0e6-43b0-bb52-72b2292f2dfd admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:52.012789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86f5ce-f0e6-43b0-bb52-72b2292f2dfd admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:52.018756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5283224-1ccc-49a0-bbb8-2b3bfc7b270b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:52.019434 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5283224-1ccc-49a0-bbb8-2b3bfc7b270b admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:52.019708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5283224-1ccc-49a0-bbb8-2b3bfc7b270b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:52.019975 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5283224-1ccc-49a0-bbb8-2b3bfc7b270b admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:52.020680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5283224-1ccc-49a0-bbb8-2b3bfc7b270b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:52.022592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5283224-1ccc-49a0-bbb8-2b3bfc7b270b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:52.055467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1321/2655] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:35:51 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:35:52.057393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a50007-5685-473b-aef9-bb5404bce001 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:52.070088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a50007-5685-473b-aef9-bb5404bce001 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:52.135259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8d0fb-a32f-4e79-9303-7915d7245a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:52.135973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8d0fb-a32f-4e79-9303-7915d7245a4d admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:52.136251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8d0fb-a32f-4e79-9303-7915d7245a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:52.136526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8d0fb-a32f-4e79-9303-7915d7245a4d admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:52.137220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8d0fb-a32f-4e79-9303-7915d7245a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:52.139197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8d0fb-a32f-4e79-9303-7915d7245a4d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:52.162127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1335/2656] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:35:52 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:35:52.166248 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5283224-1ccc-49a0-bbb8-2b3bfc7b270b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:52.178928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5283224-1ccc-49a0-bbb8-2b3bfc7b270b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:52.187954 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d5c67-c5ac-4136-a68a-ffb63b4f708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:52.188774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d5c67-c5ac-4136-a68a-ffb63b4f708d admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:52.189140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d5c67-c5ac-4136-a68a-ffb63b4f708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:52.189502 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d5c67-c5ac-4136-a68a-ffb63b4f708d admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:52.190330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d5c67-c5ac-4136-a68a-ffb63b4f708d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:52.192130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d5c67-c5ac-4136-a68a-ffb63b4f708d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:52.208751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1322/2657] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:35:52.213172 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8d0fb-a32f-4e79-9303-7915d7245a4d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:52.225394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8d0fb-a32f-4e79-9303-7915d7245a4d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:52.231206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a831f-0869-4112-9746-e9f14c04cac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:52.232103 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a831f-0869-4112-9746-e9f14c04cac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:52.232384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a831f-0869-4112-9746-e9f14c04cac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:52.232656 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a831f-0869-4112-9746-e9f14c04cac0 admin admin] PATH_INFO: `/v3/projects/3b2b5ae1ffb946a184941725e5031d8d/users/a9d5889a4f1d47578f4c642a3f707b7d/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:52.249338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a831f-0869-4112-9746-e9f14c04cac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b2b5ae1ffb946a184941725e5031d8d, user_id=a9d5889a4f1d47578f4c642a3f707b7d, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:52.252122 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a831f-0869-4112-9746-e9f14c04cac0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:52.267083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1336/2658] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:35:52 2022] PUT /identity/v3/projects/3b2b5ae1ffb946a184941725e5031d8d/users/a9d5889a4f1d47578f4c642a3f707b7d/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:52.271288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d5c67-c5ac-4136-a68a-ffb63b4f708d admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:52.282271 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d5c67-c5ac-4136-a68a-ffb63b4f708d admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:52.355212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d884c54-7f16-4580-b284-121b11764a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:52.355896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d884c54-7f16-4580-b284-121b11764a78 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:52.356181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d884c54-7f16-4580-b284-121b11764a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:52.356452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d884c54-7f16-4580-b284-121b11764a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:52.357176 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d884c54-7f16-4580-b284-121b11764a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:52.358808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d884c54-7f16-4580-b284-121b11764a78 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:52.370253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1323/2659] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:35:52.374501 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a831f-0869-4112-9746-e9f14c04cac0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:52.386842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a831f-0869-4112-9746-e9f14c04cac0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:52.392864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c0b9e-1b93-490e-91a1-5097874bcbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:52.393774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c0b9e-1b93-490e-91a1-5097874bcbd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:52.394060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c0b9e-1b93-490e-91a1-5097874bcbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:52.394338 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c0b9e-1b93-490e-91a1-5097874bcbd7 admin admin] PATH_INFO: `/v3/projects/3b2b5ae1ffb946a184941725e5031d8d/users/a9d5889a4f1d47578f4c642a3f707b7d/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:52.396451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c0b9e-1b93-490e-91a1-5097874bcbd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b2b5ae1ffb946a184941725e5031d8d, user_id=a9d5889a4f1d47578f4c642a3f707b7d, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:52.399167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c0b9e-1b93-490e-91a1-5097874bcbd7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:52.412390 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1337/2660] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:35:52 2022] PUT /identity/v3/projects/3b2b5ae1ffb946a184941725e5031d8d/users/a9d5889a4f1d47578f4c642a3f707b7d/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:35:52.423587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ebde1-ea2f-4337-9c95-ca0ad30eb1e8 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:52.423871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ebde1-ea2f-4337-9c95-ca0ad30eb1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:52.424144 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ebde1-ea2f-4337-9c95-ca0ad30eb1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:52.454872 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-699ebde1-ea2f-4337-9c95-ca0ad30eb1e8 None None] MFA Rules not processed for user `a9d5889a4f1d47578f4c642a3f707b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:35:52.456093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-699ebde1-ea2f-4337-9c95-ca0ad30eb1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:52.486131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:35:52.538733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1324/2661] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:35:52 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:35:52.551231 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c0b9e-1b93-490e-91a1-5097874bcbd7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:52.553188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5b8c0b9e-1b93-490e-91a1-5097874bcbd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:35:52.616588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c0b9e-1b93-490e-91a1-5097874bcbd7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:52.621135 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b18893-bcb2-47e0-b072-3328e6d6a989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:52.621797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b18893-bcb2-47e0-b072-3328e6d6a989 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:52.622074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b18893-bcb2-47e0-b072-3328e6d6a989 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:52.622350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b18893-bcb2-47e0-b072-3328e6d6a989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:52.633024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b18893-bcb2-47e0-b072-3328e6d6a989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:52.634482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b18893-bcb2-47e0-b072-3328e6d6a989 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:52.644948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1338/2662] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:35:52 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:35:52.725455 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ebde1-ea2f-4337-9c95-ca0ad30eb1e8 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:35:52.736425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ebde1-ea2f-4337-9c95-ca0ad30eb1e8 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:35:52.740833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac169a-bb71-4c4c-bf78-77e8b7ae9e71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:35:52.741496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac169a-bb71-4c4c-bf78-77e8b7ae9e71 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:35:52.741774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac169a-bb71-4c4c-bf78-77e8b7ae9e71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:35:52.742044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac169a-bb71-4c4c-bf78-77e8b7ae9e71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:35:52.808891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac169a-bb71-4c4c-bf78-77e8b7ae9e71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:35:52.810341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac169a-bb71-4c4c-bf78-77e8b7ae9e71 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:35:52.820545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1325/2663] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:35:52 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:02.640318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:02.815642 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=44 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:03.367689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b18893-bcb2-47e0-b072-3328e6d6a989 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:03.445513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b18893-bcb2-47e0-b072-3328e6d6a989 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:03.450536 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:03.451456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:03.451733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:03.452003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] PATH_INFO: `/v3/users/176a4cd7bb0d42c4b495464e4f4369f3` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:03.453309 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=176a4cd7bb0d42c4b495464e4f4369f3)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:03.455401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:03.511673 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176a4cd7bb0d42c4b495464e4f4369f3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:03.526883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176a4cd7bb0d42c4b495464e4f4369f3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:03.527619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176a4cd7bb0d42c4b495464e4f4369f3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:03.535206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176a4cd7bb0d42c4b495464e4f4369f3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:03.536206 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176a4cd7bb0d42c4b495464e4f4369f3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:03.549687 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1339/2664] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:03 2022] DELETE /identity/v3/users/176a4cd7bb0d42c4b495464e4f4369f3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:03.555076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fea04-b1b5-4f79-bdbd-6c88896fdaed None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:03.555363 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fea04-b1b5-4f79-bdbd-6c88896fdaed None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:03.555695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fea04-b1b5-4f79-bdbd-6c88896fdaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:03.578769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-3d7fea04-b1b5-4f79-bdbd-6c88896fdaed None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:03.580479 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3d7fea04-b1b5-4f79-bdbd-6c88896fdaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:03.644215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1326/2665] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:03.654654 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:03.656150 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:03.721604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c1f66-a1c6-4095-9185-f06957be3d39 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:03.725821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc04581-0f55-47e2-844b-148e53d895a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:03.726520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc04581-0f55-47e2-844b-148e53d895a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:03.726838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc04581-0f55-47e2-844b-148e53d895a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:03.727112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc04581-0f55-47e2-844b-148e53d895a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:03.737414 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc04581-0f55-47e2-844b-148e53d895a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:03.739145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc04581-0f55-47e2-844b-148e53d895a8 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:03.749576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1340/2666] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:04.061385 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fea04-b1b5-4f79-bdbd-6c88896fdaed None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:04.063246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3d7fea04-b1b5-4f79-bdbd-6c88896fdaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:04.076667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fea04-b1b5-4f79-bdbd-6c88896fdaed None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:04.081894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:04.083860 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:04.083860 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:04.083860 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] PATH_INFO: `/v3/projects/f13e186b878a4c3fb356ad6ee449a134` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:04.086724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f13e186b878a4c3fb356ad6ee449a134)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:04.088954 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:04.153178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f13e186b878a4c3fb356ad6ee449a134', 'request_id': 'req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.163174 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f13e186b878a4c3fb356ad6ee449a134', 'request_id': 'req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.170226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f13e186b878a4c3fb356ad6ee449a134', 'request_id': 'req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.183652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] The token cache is being invalidate because project f13e186b878a4c3fb356ad6ee449a134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:04.183904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.189804 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.191552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1327/2667] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:04 2022] DELETE /identity/v3/projects/f13e186b878a4c3fb356ad6ee449a134 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:04.195791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc04581-0f55-47e2-844b-148e53d895a8 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:04.206486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc04581-0f55-47e2-844b-148e53d895a8 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:04.228855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:04.272733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d2af5-cc79-4609-b70c-78638bf1989c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:04.273518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d2af5-cc79-4609-b70c-78638bf1989c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:04.273791 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d2af5-cc79-4609-b70c-78638bf1989c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:04.274062 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d2af5-cc79-4609-b70c-78638bf1989c admin admin] PATH_INFO: `/v3/users/a9d5889a4f1d47578f4c642a3f707b7d` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:04.275277 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d2af5-cc79-4609-b70c-78638bf1989c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9d5889a4f1d47578f4c642a3f707b7d)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:04.277219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d2af5-cc79-4609-b70c-78638bf1989c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:04.328011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c88d2af5-cc79-4609-b70c-78638bf1989c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9d5889a4f1d47578f4c642a3f707b7d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.340891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c88d2af5-cc79-4609-b70c-78638bf1989c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9d5889a4f1d47578f4c642a3f707b7d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.341652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c88d2af5-cc79-4609-b70c-78638bf1989c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9d5889a4f1d47578f4c642a3f707b7d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.348876 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c88d2af5-cc79-4609-b70c-78638bf1989c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9d5889a4f1d47578f4c642a3f707b7d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.349577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c88d2af5-cc79-4609-b70c-78638bf1989c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9d5889a4f1d47578f4c642a3f707b7d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.362607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1341/2668] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:04 2022] DELETE /identity/v3/users/a9d5889a4f1d47578f4c642a3f707b7d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:04.715303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:04.717851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:04.736909 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845da2f-8ad9-43fc-9546-753d4c8b0ea7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:04.808940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:04.809794 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:04.810075 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:04.810341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] PATH_INFO: `/v3/projects/3b2b5ae1ffb946a184941725e5031d8d` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:04.811797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b2b5ae1ffb946a184941725e5031d8d)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:04.813832 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:04.851634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:04.871574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b2b5ae1ffb946a184941725e5031d8d', 'request_id': 'req-31358df3-7be0-4c3f-ad0c-71c8e098dc52'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.880457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b2b5ae1ffb946a184941725e5031d8d', 'request_id': 'req-31358df3-7be0-4c3f-ad0c-71c8e098dc52'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.886977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b2b5ae1ffb946a184941725e5031d8d', 'request_id': 'req-31358df3-7be0-4c3f-ad0c-71c8e098dc52'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.899044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] The token cache is being invalidate because project 3b2b5ae1ffb946a184941725e5031d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:04.899313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.900034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-31358df3-7be0-4c3f-ad0c-71c8e098dc52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:04.901640 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1328/2669] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:04 2022] DELETE /identity/v3/projects/3b2b5ae1ffb946a184941725e5031d8d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:04.906959 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8152be-9fbc-4160-95a6-34488a48f947 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:04.907463 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8152be-9fbc-4160-95a6-34488a48f947 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:04.907463 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8152be-9fbc-4160-95a6-34488a48f947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:04.928913 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7a8152be-9fbc-4160-95a6-34488a48f947 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:04.930145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7a8152be-9fbc-4160-95a6-34488a48f947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:05.002024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1342/2670] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:05.010536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62afd90-a361-4ba5-a604-c48df98dc76a None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.010997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62afd90-a361-4ba5-a604-c48df98dc76a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.011404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62afd90-a361-4ba5-a604-c48df98dc76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.031921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-f62afd90-a361-4ba5-a604-c48df98dc76a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:05.033285 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f62afd90-a361-4ba5-a604-c48df98dc76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:05.042038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1329/2671] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:36:05.046554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8152be-9fbc-4160-95a6-34488a48f947 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.057631 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8152be-9fbc-4160-95a6-34488a48f947 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.062838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92e4de-2316-4dcc-a7a1-9c8d03096b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.063531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92e4de-2316-4dcc-a7a1-9c8d03096b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.063889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92e4de-2316-4dcc-a7a1-9c8d03096b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.064226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92e4de-2316-4dcc-a7a1-9c8d03096b5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.064989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92e4de-2316-4dcc-a7a1-9c8d03096b5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:36:05.065316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92e4de-2316-4dcc-a7a1-9c8d03096b5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.067249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92e4de-2316-4dcc-a7a1-9c8d03096b5c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.080850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1343/2672] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:36:05.086866 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62afd90-a361-4ba5-a604-c48df98dc76a None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.096885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62afd90-a361-4ba5-a604-c48df98dc76a None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.101698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962159a-e86b-4828-af98-b08af057cb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.102382 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962159a-e86b-4828-af98-b08af057cb88 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.102699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962159a-e86b-4828-af98-b08af057cb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.102972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962159a-e86b-4828-af98-b08af057cb88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.103680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962159a-e86b-4828-af98-b08af057cb88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.105587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962159a-e86b-4828-af98-b08af057cb88 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.134621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1330/2673] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:36:05 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:05.138037 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92e4de-2316-4dcc-a7a1-9c8d03096b5c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.148797 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92e4de-2316-4dcc-a7a1-9c8d03096b5c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.177845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:05.212992 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb1e9-e52d-44bf-9585-120626a9ae1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.213764 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb1e9-e52d-44bf-9585-120626a9ae1e admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.214129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb1e9-e52d-44bf-9585-120626a9ae1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.214473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb1e9-e52d-44bf-9585-120626a9ae1e admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.215331 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cb1e9-e52d-44bf-9585-120626a9ae1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.217396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cb1e9-e52d-44bf-9585-120626a9ae1e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.239353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1344/2674] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:36:05 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:05.243617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962159a-e86b-4828-af98-b08af057cb88 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.255511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962159a-e86b-4828-af98-b08af057cb88 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.260357 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262e425-cb47-46ab-bbf0-27de27584f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.261071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262e425-cb47-46ab-bbf0-27de27584f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.261349 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262e425-cb47-46ab-bbf0-27de27584f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.261621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262e425-cb47-46ab-bbf0-27de27584f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.262381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262e425-cb47-46ab-bbf0-27de27584f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.264065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262e425-cb47-46ab-bbf0-27de27584f75 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.282713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1331/2675] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 10:36:05.283953 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb1e9-e52d-44bf-9585-120626a9ae1e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.295279 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb1e9-e52d-44bf-9585-120626a9ae1e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.299958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db564cb5-6254-48a9-80fc-9eb43e26f1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.300761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db564cb5-6254-48a9-80fc-9eb43e26f1eb admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.301684 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db564cb5-6254-48a9-80fc-9eb43e26f1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.301963 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db564cb5-6254-48a9-80fc-9eb43e26f1eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.302739 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db564cb5-6254-48a9-80fc-9eb43e26f1eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.304380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db564cb5-6254-48a9-80fc-9eb43e26f1eb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.316840 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1345/2676] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:05.321261 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262e425-cb47-46ab-bbf0-27de27584f75 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.331856 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262e425-cb47-46ab-bbf0-27de27584f75 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.337373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412af4d-1d89-44ff-bfdf-6f0021a3dbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.338287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412af4d-1d89-44ff-bfdf-6f0021a3dbcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.338575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412af4d-1d89-44ff-bfdf-6f0021a3dbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.339149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412af4d-1d89-44ff-bfdf-6f0021a3dbcb admin admin] PATH_INFO: `/v3/projects/a5a2d93a002443148c22eb1518b4e3ea/users/ac91973138364b1e8b4643c86c6157bd/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.356837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412af4d-1d89-44ff-bfdf-6f0021a3dbcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a2d93a002443148c22eb1518b4e3ea, user_id=ac91973138364b1e8b4643c86c6157bd, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.359585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412af4d-1d89-44ff-bfdf-6f0021a3dbcb admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.375900 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1332/2677] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:05 2022] PUT /identity/v3/projects/a5a2d93a002443148c22eb1518b4e3ea/users/ac91973138364b1e8b4643c86c6157bd/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:36:05.387374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa0f01-86c1-46f7-8466-f36ebaae97fa None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.387628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa0f01-86c1-46f7-8466-f36ebaae97fa None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.387918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa0f01-86c1-46f7-8466-f36ebaae97fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.422762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f9fa0f01-86c1-46f7-8466-f36ebaae97fa None None] MFA Rules not processed for user `ac91973138364b1e8b4643c86c6157bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:05.424457 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f9fa0f01-86c1-46f7-8466-f36ebaae97fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:05.497837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1346/2678] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:05.502737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412af4d-1d89-44ff-bfdf-6f0021a3dbcb admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.571496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412af4d-1d89-44ff-bfdf-6f0021a3dbcb admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.576591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f778183-703c-4e7e-9df6-7a993bd2521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.577387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f778183-703c-4e7e-9df6-7a993bd2521f admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.577667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f778183-703c-4e7e-9df6-7a993bd2521f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.577944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f778183-703c-4e7e-9df6-7a993bd2521f admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.578665 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f778183-703c-4e7e-9df6-7a993bd2521f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.580599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f778183-703c-4e7e-9df6-7a993bd2521f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.615262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1333/2679] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:36:05 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:05.619704 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa0f01-86c1-46f7-8466-f36ebaae97fa None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.631038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa0f01-86c1-46f7-8466-f36ebaae97fa None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.711502 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3749f-a8a8-407e-8a26-d1f2ea2989c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.712191 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3749f-a8a8-407e-8a26-d1f2ea2989c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.712473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3749f-a8a8-407e-8a26-d1f2ea2989c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.712749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3749f-a8a8-407e-8a26-d1f2ea2989c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.713461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c3749f-a8a8-407e-8a26-d1f2ea2989c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.715440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c3749f-a8a8-407e-8a26-d1f2ea2989c9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.739326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1347/2680] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:36:05 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:36:05.742671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f778183-703c-4e7e-9df6-7a993bd2521f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.753920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f778183-703c-4e7e-9df6-7a993bd2521f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.760432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31755938-d221-4261-9a70-422320731a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.761120 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755938-d221-4261-9a70-422320731a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.761390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755938-d221-4261-9a70-422320731a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.761652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755938-d221-4261-9a70-422320731a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.762388 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31755938-d221-4261-9a70-422320731a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.764042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31755938-d221-4261-9a70-422320731a23 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.776662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1334/2681] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:05.781425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3749f-a8a8-407e-8a26-d1f2ea2989c9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.792266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3749f-a8a8-407e-8a26-d1f2ea2989c9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.797301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62382d-0343-4899-b3cf-7823e7dff0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.798212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62382d-0343-4899-b3cf-7823e7dff0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.798492 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62382d-0343-4899-b3cf-7823e7dff0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.798809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62382d-0343-4899-b3cf-7823e7dff0f4 admin admin] PATH_INFO: `/v3/projects/296371e6645d477593122b20b73601a8/users/a6bfba7f1b384cffa8f46bb9a3ac3372/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.816044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f62382d-0343-4899-b3cf-7823e7dff0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296371e6645d477593122b20b73601a8, user_id=a6bfba7f1b384cffa8f46bb9a3ac3372, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.819603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f62382d-0343-4899-b3cf-7823e7dff0f4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.836050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1348/2682] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:05 2022] PUT /identity/v3/projects/296371e6645d477593122b20b73601a8/users/a6bfba7f1b384cffa8f46bb9a3ac3372/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:05.840206 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31755938-d221-4261-9a70-422320731a23 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.851220 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31755938-d221-4261-9a70-422320731a23 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.874725 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:05.925621 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc8f6a-9f3e-4df4-80b1-d1e223dc1e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.927027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc8f6a-9f3e-4df4-80b1-d1e223dc1e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.927469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc8f6a-9f3e-4df4-80b1-d1e223dc1e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.927896 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc8f6a-9f3e-4df4-80b1-d1e223dc1e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.928662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc8f6a-9f3e-4df4-80b1-d1e223dc1e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.930290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc8f6a-9f3e-4df4-80b1-d1e223dc1e60 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.942713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1335/2683] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:05.947051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62382d-0343-4899-b3cf-7823e7dff0f4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:05.953139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:05.959372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62382d-0343-4899-b3cf-7823e7dff0f4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:05.965835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2567643-de11-47ee-a180-62e3778df0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:05.966792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2567643-de11-47ee-a180-62e3778df0e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:05.967104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2567643-de11-47ee-a180-62e3778df0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:05.967379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2567643-de11-47ee-a180-62e3778df0e6 admin admin] PATH_INFO: `/v3/projects/296371e6645d477593122b20b73601a8/users/a6bfba7f1b384cffa8f46bb9a3ac3372/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:05.969546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2567643-de11-47ee-a180-62e3778df0e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296371e6645d477593122b20b73601a8, user_id=a6bfba7f1b384cffa8f46bb9a3ac3372, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:05.972270 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2567643-de11-47ee-a180-62e3778df0e6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:05.987026 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1349/2684] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:05 2022] PUT /identity/v3/projects/296371e6645d477593122b20b73601a8/users/a6bfba7f1b384cffa8f46bb9a3ac3372/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:06.059343 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53975329-105d-41b8-a6cf-5df7df854971 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:06.059567 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53975329-105d-41b8-a6cf-5df7df854971 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:06.059871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53975329-105d-41b8-a6cf-5df7df854971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:06.089754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-53975329-105d-41b8-a6cf-5df7df854971 None None] MFA Rules not processed for user `a6bfba7f1b384cffa8f46bb9a3ac3372`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:06.090919 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-53975329-105d-41b8-a6cf-5df7df854971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:06.167685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1336/2685] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:06 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:06.180441 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2567643-de11-47ee-a180-62e3778df0e6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:06.182844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b2567643-de11-47ee-a180-62e3778df0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:06.258404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2567643-de11-47ee-a180-62e3778df0e6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:06.264906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3b76d-496a-4f95-93e6-2d77e47cf6fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:06.265575 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3b76d-496a-4f95-93e6-2d77e47cf6fe service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:06.265855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3b76d-496a-4f95-93e6-2d77e47cf6fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:06.266123 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3b76d-496a-4f95-93e6-2d77e47cf6fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:06.277945 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3b76d-496a-4f95-93e6-2d77e47cf6fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:06.279468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3b76d-496a-4f95-93e6-2d77e47cf6fe service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:06.290257 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1350/2686] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:06 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:06.371023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53975329-105d-41b8-a6cf-5df7df854971 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:06.382479 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53975329-105d-41b8-a6cf-5df7df854971 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:06.386966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63e92a-0ba8-4227-89cc-af00b40dc573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:06.387639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63e92a-0ba8-4227-89cc-af00b40dc573 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:06.387921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63e92a-0ba8-4227-89cc-af00b40dc573 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:06.388193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63e92a-0ba8-4227-89cc-af00b40dc573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:06.460791 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63e92a-0ba8-4227-89cc-af00b40dc573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:06.462258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63e92a-0ba8-4227-89cc-af00b40dc573 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:06.472626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1337/2687] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:09.255393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3b76d-496a-4f95-93e6-2d77e47cf6fe service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:09.340935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3b76d-496a-4f95-93e6-2d77e47cf6fe service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:09.348088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:09.349476 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:09.349935 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:09.350384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] PATH_INFO: `/v3/users/ac91973138364b1e8b4643c86c6157bd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:09.352164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac91973138364b1e8b4643c86c6157bd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:09.355549 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:09.412362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac91973138364b1e8b4643c86c6157bd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:09.424606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac91973138364b1e8b4643c86c6157bd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:09.425498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac91973138364b1e8b4643c86c6157bd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:09.433028 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac91973138364b1e8b4643c86c6157bd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:09.433699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac91973138364b1e8b4643c86c6157bd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:09.449115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1351/2688] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:09 2022] DELETE /identity/v3/users/ac91973138364b1e8b4643c86c6157bd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:09.453837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4beae-0f45-4712-ba17-38e6ee2053c9 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:09.454377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4beae-0f45-4712-ba17-38e6ee2053c9 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:09.454377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4beae-0f45-4712-ba17-38e6ee2053c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:09.474402 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-7be4beae-0f45-4712-ba17-38e6ee2053c9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:09.475885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7be4beae-0f45-4712-ba17-38e6ee2053c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:09.522398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=100 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:09.543202 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1338/2689] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:09.553961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:09.555650 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:09.572504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:09.624019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611fd24-27b1-4b4a-87f6-9237bb147c61 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:09.629629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1dd90-cb1a-4d65-99bb-d5972d6f68c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:09.630768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1dd90-cb1a-4d65-99bb-d5972d6f68c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:09.631194 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1dd90-cb1a-4d65-99bb-d5972d6f68c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:09.631648 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1dd90-cb1a-4d65-99bb-d5972d6f68c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:09.647036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1dd90-cb1a-4d65-99bb-d5972d6f68c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:09.648517 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1dd90-cb1a-4d65-99bb-d5972d6f68c8 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:09.660305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1352/2690] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:09.996935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4beae-0f45-4712-ba17-38e6ee2053c9 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:09.999172 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7be4beae-0f45-4712-ba17-38e6ee2053c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:10.018165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4beae-0f45-4712-ba17-38e6ee2053c9 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:10.024931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:10.025985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:10.026387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:10.026853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] PATH_INFO: `/v3/projects/a5a2d93a002443148c22eb1518b4e3ea` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:10.028683 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a2d93a002443148c22eb1518b4e3ea)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:10.031264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:10.100766 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a2d93a002443148c22eb1518b4e3ea', 'request_id': 'req-5345714e-0b62-4d2d-a139-8238768964cc'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.110199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a2d93a002443148c22eb1518b4e3ea', 'request_id': 'req-5345714e-0b62-4d2d-a139-8238768964cc'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.119191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a2d93a002443148c22eb1518b4e3ea', 'request_id': 'req-5345714e-0b62-4d2d-a139-8238768964cc'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.137876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] The token cache is being invalidate because project a5a2d93a002443148c22eb1518b4e3ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:10.138334 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.139561 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.142425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1339/2691] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:09 2022] DELETE /identity/v3/projects/a5a2d93a002443148c22eb1518b4e3ea => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:10.147095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1dd90-cb1a-4d65-99bb-d5972d6f68c8 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:10.157875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1dd90-cb1a-4d65-99bb-d5972d6f68c8 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:10.249898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6315c3-f2e6-4e55-907f-d5f0196d7166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:10.250822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6315c3-f2e6-4e55-907f-d5f0196d7166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:10.251139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6315c3-f2e6-4e55-907f-d5f0196d7166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:10.251398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6315c3-f2e6-4e55-907f-d5f0196d7166 admin admin] PATH_INFO: `/v3/users/a6bfba7f1b384cffa8f46bb9a3ac3372` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:10.252685 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6315c3-f2e6-4e55-907f-d5f0196d7166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6bfba7f1b384cffa8f46bb9a3ac3372)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:10.254813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6315c3-f2e6-4e55-907f-d5f0196d7166 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:10.308352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8d6315c3-f2e6-4e55-907f-d5f0196d7166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6bfba7f1b384cffa8f46bb9a3ac3372'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.323071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8d6315c3-f2e6-4e55-907f-d5f0196d7166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6bfba7f1b384cffa8f46bb9a3ac3372'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.323911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8d6315c3-f2e6-4e55-907f-d5f0196d7166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6bfba7f1b384cffa8f46bb9a3ac3372'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.331772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8d6315c3-f2e6-4e55-907f-d5f0196d7166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6bfba7f1b384cffa8f46bb9a3ac3372'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.332477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8d6315c3-f2e6-4e55-907f-d5f0196d7166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6bfba7f1b384cffa8f46bb9a3ac3372'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.348868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1353/2692] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:10 2022] DELETE /identity/v3/users/a6bfba7f1b384cffa8f46bb9a3ac3372 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:10.710678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:10.712697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:10.727149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345714e-0b62-4d2d-a139-8238768964cc admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:10.803494 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:10.804399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:10.804680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:10.804954 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] PATH_INFO: `/v3/projects/296371e6645d477593122b20b73601a8` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:10.806590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] RBAC: Authorizing `identity:delete_project(project_id=296371e6645d477593122b20b73601a8)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:10.808577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:10.870491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296371e6645d477593122b20b73601a8', 'request_id': 'req-9915c1f8-eb61-4bf1-9a23-bca5671b59be'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.881398 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296371e6645d477593122b20b73601a8', 'request_id': 'req-9915c1f8-eb61-4bf1-9a23-bca5671b59be'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.891137 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296371e6645d477593122b20b73601a8', 'request_id': 'req-9915c1f8-eb61-4bf1-9a23-bca5671b59be'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.903641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:10.912586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] The token cache is being invalidate because project 296371e6645d477593122b20b73601a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:10.913122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.914550 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-9915c1f8-eb61-4bf1-9a23-bca5671b59be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:10.918042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1340/2693] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:10 2022] DELETE /identity/v3/projects/296371e6645d477593122b20b73601a8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:10.925786 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472980b-10d8-4532-b748-7c76d2d5688f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:10.926292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472980b-10d8-4532-b748-7c76d2d5688f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:10.926774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472980b-10d8-4532-b748-7c76d2d5688f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:10.954475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5472980b-10d8-4532-b748-7c76d2d5688f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:10.958468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5472980b-10d8-4532-b748-7c76d2d5688f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:10.993906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:11.035981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1354/2694] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:11.046058 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b585a7-143d-4cdb-938e-7d2abf4a1a43 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.046344 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b585a7-143d-4cdb-938e-7d2abf4a1a43 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.046681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b585a7-143d-4cdb-938e-7d2abf4a1a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.067348 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-f0b585a7-143d-4cdb-938e-7d2abf4a1a43 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:11.068588 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f0b585a7-143d-4cdb-938e-7d2abf4a1a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:11.080647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1341/2695] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:36:11.085695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472980b-10d8-4532-b748-7c76d2d5688f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:11.096918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472980b-10d8-4532-b748-7c76d2d5688f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:11.103339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da543ddc-ee26-4fd0-8e9d-911f7b8ee199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:11.104027 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da543ddc-ee26-4fd0-8e9d-911f7b8ee199 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.104303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da543ddc-ee26-4fd0-8e9d-911f7b8ee199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.104574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da543ddc-ee26-4fd0-8e9d-911f7b8ee199 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.105307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da543ddc-ee26-4fd0-8e9d-911f7b8ee199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:36:11.105609 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da543ddc-ee26-4fd0-8e9d-911f7b8ee199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:11.107402 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da543ddc-ee26-4fd0-8e9d-911f7b8ee199 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:11.122179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1355/2696] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:36:11.129448 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b585a7-143d-4cdb-938e-7d2abf4a1a43 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:11.140587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b585a7-143d-4cdb-938e-7d2abf4a1a43 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:11.146500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a835cc-6128-4f71-ae3d-9d549f832160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:11.147315 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a835cc-6128-4f71-ae3d-9d549f832160 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.147599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a835cc-6128-4f71-ae3d-9d549f832160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.147874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a835cc-6128-4f71-ae3d-9d549f832160 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.148599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a835cc-6128-4f71-ae3d-9d549f832160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:11.150596 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a835cc-6128-4f71-ae3d-9d549f832160 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:11.182734 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1342/2697] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:36:11 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:11.186974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da543ddc-ee26-4fd0-8e9d-911f7b8ee199 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:11.198296 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da543ddc-ee26-4fd0-8e9d-911f7b8ee199 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:11.261428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddeb01-afab-49f9-bf50-b0a3aace27bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:11.262050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddeb01-afab-49f9-bf50-b0a3aace27bc admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.262339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddeb01-afab-49f9-bf50-b0a3aace27bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.262658 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddeb01-afab-49f9-bf50-b0a3aace27bc admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.263367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fddeb01-afab-49f9-bf50-b0a3aace27bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:11.265339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fddeb01-afab-49f9-bf50-b0a3aace27bc admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:11.289693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1356/2698] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:36:11 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:11.293887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a835cc-6128-4f71-ae3d-9d549f832160 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:11.306334 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a835cc-6128-4f71-ae3d-9d549f832160 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:11.311733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e114-ebf8-46f2-8da0-ed5af5925cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:11.312575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e114-ebf8-46f2-8da0-ed5af5925cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.312916 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e114-ebf8-46f2-8da0-ed5af5925cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.313250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e114-ebf8-46f2-8da0-ed5af5925cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.314073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955e114-ebf8-46f2-8da0-ed5af5925cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:11.315938 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955e114-ebf8-46f2-8da0-ed5af5925cd4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:11.329059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1343/2699] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:11.334242 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddeb01-afab-49f9-bf50-b0a3aace27bc admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:11.348046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddeb01-afab-49f9-bf50-b0a3aace27bc admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:11.355647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625c161-aa17-466c-9543-ac9165e0d740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:11.356571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625c161-aa17-466c-9543-ac9165e0d740 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.356571 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625c161-aa17-466c-9543-ac9165e0d740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.356892 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625c161-aa17-466c-9543-ac9165e0d740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.360108 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625c161-aa17-466c-9543-ac9165e0d740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:11.361927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625c161-aa17-466c-9543-ac9165e0d740 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:11.374774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1357/2700] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:11.379473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e114-ebf8-46f2-8da0-ed5af5925cd4 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:11.391829 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e114-ebf8-46f2-8da0-ed5af5925cd4 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:11.396747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347e54a-9fc5-4548-a943-e6752034cafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:11.397649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347e54a-9fc5-4548-a943-e6752034cafa admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.398094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347e54a-9fc5-4548-a943-e6752034cafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.398430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347e54a-9fc5-4548-a943-e6752034cafa admin admin] PATH_INFO: `/v3/projects/c235c408a039405189d241fdbac52ae2/users/0294ce923aa64269ac1c4307f26e88fa/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.418494 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347e54a-9fc5-4548-a943-e6752034cafa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c235c408a039405189d241fdbac52ae2, user_id=0294ce923aa64269ac1c4307f26e88fa, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:11.423380 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347e54a-9fc5-4548-a943-e6752034cafa admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:11.440592 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1344/2701] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:11 2022] PUT /identity/v3/projects/c235c408a039405189d241fdbac52ae2/users/0294ce923aa64269ac1c4307f26e88fa/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:11.457385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f265a8-5d79-41a0-b322-6b663602a695 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.457861 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f265a8-5d79-41a0-b322-6b663602a695 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.458308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f265a8-5d79-41a0-b322-6b663602a695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.497568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-62f265a8-5d79-41a0-b322-6b663602a695 None None] MFA Rules not processed for user `0294ce923aa64269ac1c4307f26e88fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:11.498840 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-62f265a8-5d79-41a0-b322-6b663602a695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:11.580350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1358/2702] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:36:11.587789 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347e54a-9fc5-4548-a943-e6752034cafa admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:11.652669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347e54a-9fc5-4548-a943-e6752034cafa admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:11.657739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c575b-a8bf-4186-8e5e-f56d1b7dd6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:11.658432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c575b-a8bf-4186-8e5e-f56d1b7dd6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.658746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c575b-a8bf-4186-8e5e-f56d1b7dd6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.659017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c575b-a8bf-4186-8e5e-f56d1b7dd6d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.659724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c575b-a8bf-4186-8e5e-f56d1b7dd6d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:11.661696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c575b-a8bf-4186-8e5e-f56d1b7dd6d7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:11.692600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1345/2703] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:36:11 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:36:11.697555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f265a8-5d79-41a0-b322-6b663602a695 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:11.709590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f265a8-5d79-41a0-b322-6b663602a695 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:11.740922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:11.780232 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f85f1b-fa4b-414e-9941-b3267b926e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:11.780920 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f85f1b-fa4b-414e-9941-b3267b926e25 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.781198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f85f1b-fa4b-414e-9941-b3267b926e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.781468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f85f1b-fa4b-414e-9941-b3267b926e25 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.782166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f85f1b-fa4b-414e-9941-b3267b926e25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:11.784184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f85f1b-fa4b-414e-9941-b3267b926e25 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:11.808784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1359/2704] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:36:11 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:11.813743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c575b-a8bf-4186-8e5e-f56d1b7dd6d7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:11.825291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c575b-a8bf-4186-8e5e-f56d1b7dd6d7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:11.831055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7bd00-a8fa-4d22-8e9d-675e26e85556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:11.831710 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7bd00-a8fa-4d22-8e9d-675e26e85556 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.832667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7bd00-a8fa-4d22-8e9d-675e26e85556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.832908 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7bd00-a8fa-4d22-8e9d-675e26e85556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.833669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7bd00-a8fa-4d22-8e9d-675e26e85556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:11.835281 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7bd00-a8fa-4d22-8e9d-675e26e85556 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:11.847064 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1346/2705] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:11.852943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f85f1b-fa4b-414e-9941-b3267b926e25 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:11.865014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f85f1b-fa4b-414e-9941-b3267b926e25 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:11.869850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d928ff-0415-45c2-bead-d0bd10fe7d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:11.870787 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d928ff-0415-45c2-bead-d0bd10fe7d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:11.871066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d928ff-0415-45c2-bead-d0bd10fe7d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:11.871339 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d928ff-0415-45c2-bead-d0bd10fe7d8f admin admin] PATH_INFO: `/v3/projects/3f0bca8207964038892a8938cf6e6a4a/users/455d039dd7e94051a20b9aecc6983f0c/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:11.888364 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d928ff-0415-45c2-bead-d0bd10fe7d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f0bca8207964038892a8938cf6e6a4a, user_id=455d039dd7e94051a20b9aecc6983f0c, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:11.891096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d928ff-0415-45c2-bead-d0bd10fe7d8f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:11.913035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1360/2706] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:11 2022] PUT /identity/v3/projects/3f0bca8207964038892a8938cf6e6a4a/users/455d039dd7e94051a20b9aecc6983f0c/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:11.917727 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7bd00-a8fa-4d22-8e9d-675e26e85556 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:11.933149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7bd00-a8fa-4d22-8e9d-675e26e85556 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:12.001147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eeb5f0-216f-4deb-8a28-49273322db70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:12.001828 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb5f0-216f-4deb-8a28-49273322db70 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:12.002109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb5f0-216f-4deb-8a28-49273322db70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:12.002384 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb5f0-216f-4deb-8a28-49273322db70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:12.003148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eeb5f0-216f-4deb-8a28-49273322db70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:12.005017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eeb5f0-216f-4deb-8a28-49273322db70 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:12.015831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1347/2707] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:12.021134 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d928ff-0415-45c2-bead-d0bd10fe7d8f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:12.039587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d928ff-0415-45c2-bead-d0bd10fe7d8f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:12.046613 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422dbaa-0c18-4878-ba8e-9369413fce14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:12.048146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422dbaa-0c18-4878-ba8e-9369413fce14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:12.048483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422dbaa-0c18-4878-ba8e-9369413fce14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:12.048760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422dbaa-0c18-4878-ba8e-9369413fce14 admin admin] PATH_INFO: `/v3/projects/3f0bca8207964038892a8938cf6e6a4a/users/455d039dd7e94051a20b9aecc6983f0c/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:12.051869 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422dbaa-0c18-4878-ba8e-9369413fce14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f0bca8207964038892a8938cf6e6a4a, user_id=455d039dd7e94051a20b9aecc6983f0c, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:12.057803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422dbaa-0c18-4878-ba8e-9369413fce14 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:12.077146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1361/2708] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:12 2022] PUT /identity/v3/projects/3f0bca8207964038892a8938cf6e6a4a/users/455d039dd7e94051a20b9aecc6983f0c/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:12.089678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb047ca1-fe0f-4184-b99d-865eb48f6735 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:12.089899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb047ca1-fe0f-4184-b99d-865eb48f6735 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:12.090142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb047ca1-fe0f-4184-b99d-865eb48f6735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:12.097279 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:12.130774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-cb047ca1-fe0f-4184-b99d-865eb48f6735 None None] MFA Rules not processed for user `455d039dd7e94051a20b9aecc6983f0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:12.132608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cb047ca1-fe0f-4184-b99d-865eb48f6735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:12.220992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1348/2709] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:12.233918 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422dbaa-0c18-4878-ba8e-9369413fce14 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:12.235792 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8422dbaa-0c18-4878-ba8e-9369413fce14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:12.311019 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422dbaa-0c18-4878-ba8e-9369413fce14 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:12.314839 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c6614-d595-4b37-91d9-fa60cf301bd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:12.315500 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c6614-d595-4b37-91d9-fa60cf301bd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:12.315778 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c6614-d595-4b37-91d9-fa60cf301bd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:12.316092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c6614-d595-4b37-91d9-fa60cf301bd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:12.326962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c6614-d595-4b37-91d9-fa60cf301bd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:12.328427 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c6614-d595-4b37-91d9-fa60cf301bd5 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:12.342212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1362/2710] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:12.411001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb047ca1-fe0f-4184-b99d-865eb48f6735 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:12.478903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb047ca1-fe0f-4184-b99d-865eb48f6735 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:12.486985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:12.488003 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:12.488342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:12.488882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] PATH_INFO: `/v3/users/0294ce923aa64269ac1c4307f26e88fa` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:12.490381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0294ce923aa64269ac1c4307f26e88fa)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:12.492567 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:12.552001 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0294ce923aa64269ac1c4307f26e88fa'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:12.569050 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0294ce923aa64269ac1c4307f26e88fa'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:12.569822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0294ce923aa64269ac1c4307f26e88fa'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:12.570425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0294ce923aa64269ac1c4307f26e88fa'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:12.578286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0294ce923aa64269ac1c4307f26e88fa'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:12.592073 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1349/2711] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:12 2022] DELETE /identity/v3/users/0294ce923aa64269ac1c4307f26e88fa => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:12.596815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0f1e-ae7f-482c-9750-bf5228deadd9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:12.597048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0f1e-ae7f-482c-9750-bf5228deadd9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:12.597459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0f1e-ae7f-482c-9750-bf5228deadd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:12.616038 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1b9d0f1e-ae7f-482c-9750-bf5228deadd9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:12.617267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1b9d0f1e-ae7f-482c-9750-bf5228deadd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:12.693720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1363/2712] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:36:12.705179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:12.707322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:12.719150 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:12.773929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957d2b0-cb43-4043-86ce-4c9da033f158 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:12.778221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74decf1-58b1-4979-b8c2-fa88736452e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:12.779131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74decf1-58b1-4979-b8c2-fa88736452e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:12.779493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74decf1-58b1-4979-b8c2-fa88736452e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:12.779772 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74decf1-58b1-4979-b8c2-fa88736452e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:12.797132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74decf1-58b1-4979-b8c2-fa88736452e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:12.798630 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74decf1-58b1-4979-b8c2-fa88736452e1 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:12.810505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1350/2713] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:13.109035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d0f1e-ae7f-482c-9750-bf5228deadd9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:13.110611 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1b9d0f1e-ae7f-482c-9750-bf5228deadd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:13.124158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d0f1e-ae7f-482c-9750-bf5228deadd9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:13.128770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:13.129768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:13.130149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:13.130523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] PATH_INFO: `/v3/projects/c235c408a039405189d241fdbac52ae2` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:13.131898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c235c408a039405189d241fdbac52ae2)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:13.133916 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:13.158453 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:13.193555 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c235c408a039405189d241fdbac52ae2', 'request_id': 'req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.200851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c235c408a039405189d241fdbac52ae2', 'request_id': 'req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.207682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c235c408a039405189d241fdbac52ae2', 'request_id': 'req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.219751 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] The token cache is being invalidate because project c235c408a039405189d241fdbac52ae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:13.220047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.220783 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.222855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1364/2714] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:13 2022] DELETE /identity/v3/projects/c235c408a039405189d241fdbac52ae2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:13.227052 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74decf1-58b1-4979-b8c2-fa88736452e1 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:13.237108 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74decf1-58b1-4979-b8c2-fa88736452e1 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:13.298018 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb393a86-999f-4046-a929-25fd3a30cc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:13.298989 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393a86-999f-4046-a929-25fd3a30cc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:13.299338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393a86-999f-4046-a929-25fd3a30cc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:13.299525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393a86-999f-4046-a929-25fd3a30cc1d admin admin] PATH_INFO: `/v3/users/455d039dd7e94051a20b9aecc6983f0c` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:13.300835 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb393a86-999f-4046-a929-25fd3a30cc1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=455d039dd7e94051a20b9aecc6983f0c)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:13.302903 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb393a86-999f-4046-a929-25fd3a30cc1d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:13.352469 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-bb393a86-999f-4046-a929-25fd3a30cc1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455d039dd7e94051a20b9aecc6983f0c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.365095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-bb393a86-999f-4046-a929-25fd3a30cc1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455d039dd7e94051a20b9aecc6983f0c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.365842 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-bb393a86-999f-4046-a929-25fd3a30cc1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455d039dd7e94051a20b9aecc6983f0c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.366492 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-bb393a86-999f-4046-a929-25fd3a30cc1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455d039dd7e94051a20b9aecc6983f0c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.373429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-bb393a86-999f-4046-a929-25fd3a30cc1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455d039dd7e94051a20b9aecc6983f0c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.387931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1351/2715] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:13 2022] DELETE /identity/v3/users/455d039dd7e94051a20b9aecc6983f0c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:13.676497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:13.678432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:13.693788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a72423-f0d9-44e6-b5ff-2c984bbd463d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:13.765057 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:13.766128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:13.766415 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:13.766719 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] PATH_INFO: `/v3/projects/3f0bca8207964038892a8938cf6e6a4a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:13.768003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f0bca8207964038892a8938cf6e6a4a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:13.770305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:13.824316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f0bca8207964038892a8938cf6e6a4a', 'request_id': 'req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.833133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f0bca8207964038892a8938cf6e6a4a', 'request_id': 'req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.840852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f0bca8207964038892a8938cf6e6a4a', 'request_id': 'req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.854817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] The token cache is being invalidate because project 3f0bca8207964038892a8938cf6e6a4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:13.855145 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.855842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-098b5f4d-27ea-424c-90c3-ba4ce2b430ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:13.857630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1365/2716] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:13 2022] DELETE /identity/v3/projects/3f0bca8207964038892a8938cf6e6a4a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:13.862639 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4910e8-6718-4dd7-b793-4e0bfedc0d76 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:13.863016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4910e8-6718-4dd7-b793-4e0bfedc0d76 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:13.863340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4910e8-6718-4dd7-b793-4e0bfedc0d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:13.883415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-8f4910e8-6718-4dd7-b793-4e0bfedc0d76 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:13.884731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8f4910e8-6718-4dd7-b793-4e0bfedc0d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:13.916432 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:13.954048 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1352/2717] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:13.964117 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd03db-402b-4d61-a3fd-b2ec8b7d76e9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:13.964521 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd03db-402b-4d61-a3fd-b2ec8b7d76e9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:13.964883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd03db-402b-4d61-a3fd-b2ec8b7d76e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:13.986305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dffd03db-402b-4d61-a3fd-b2ec8b7d76e9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:13.987582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dffd03db-402b-4d61-a3fd-b2ec8b7d76e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:13.999736 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1366/2718] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:14.004689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4910e8-6718-4dd7-b793-4e0bfedc0d76 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.015421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4910e8-6718-4dd7-b793-4e0bfedc0d76 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.020768 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6903a5-2f8e-462c-a443-df8e76f3e224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.021442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6903a5-2f8e-462c-a443-df8e76f3e224 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.021722 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6903a5-2f8e-462c-a443-df8e76f3e224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.021995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6903a5-2f8e-462c-a443-df8e76f3e224 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.022762 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6903a5-2f8e-462c-a443-df8e76f3e224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:36:14.023071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6903a5-2f8e-462c-a443-df8e76f3e224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.024688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6903a5-2f8e-462c-a443-df8e76f3e224 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.037047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1353/2719] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:36:14.041615 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd03db-402b-4d61-a3fd-b2ec8b7d76e9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.051634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd03db-402b-4d61-a3fd-b2ec8b7d76e9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.057681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56bb2b-e4b6-43ee-9b7d-d02b2ac3a2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.058396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56bb2b-e4b6-43ee-9b7d-d02b2ac3a2ac admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.058713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56bb2b-e4b6-43ee-9b7d-d02b2ac3a2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.058988 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56bb2b-e4b6-43ee-9b7d-d02b2ac3a2ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.059694 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56bb2b-e4b6-43ee-9b7d-d02b2ac3a2ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.061671 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56bb2b-e4b6-43ee-9b7d-d02b2ac3a2ac admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.098950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1367/2720] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:36:14 2022] POST /identity/v3/projects => generated 409 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:14.103544 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6903a5-2f8e-462c-a443-df8e76f3e224 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.113874 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6903a5-2f8e-462c-a443-df8e76f3e224 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.207240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41353e-961f-48ca-9c45-de57969ddc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.207940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41353e-961f-48ca-9c45-de57969ddc2e admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.208222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41353e-961f-48ca-9c45-de57969ddc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.208499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41353e-961f-48ca-9c45-de57969ddc2e admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.209222 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41353e-961f-48ca-9c45-de57969ddc2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.211229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41353e-961f-48ca-9c45-de57969ddc2e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.239567 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1354/2721] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:36:14 2022] POST /identity/v3/users => generated 305 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:36:14.246902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56bb2b-e4b6-43ee-9b7d-d02b2ac3a2ac admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.259261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56bb2b-e4b6-43ee-9b7d-d02b2ac3a2ac admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.264585 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb9268-d52f-46d1-9d8f-e1d037b645c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.265273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb9268-d52f-46d1-9d8f-e1d037b645c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.265613 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb9268-d52f-46d1-9d8f-e1d037b645c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.266138 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb9268-d52f-46d1-9d8f-e1d037b645c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.266934 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb9268-d52f-46d1-9d8f-e1d037b645c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.268626 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb9268-d52f-46d1-9d8f-e1d037b645c8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.271456 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:14.283029 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1368/2722] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:14.287145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41353e-961f-48ca-9c45-de57969ddc2e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.297908 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41353e-961f-48ca-9c45-de57969ddc2e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.305695 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b6dba-76b8-46e0-b820-5d5b549e8237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.306808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b6dba-76b8-46e0-b820-5d5b549e8237 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.307111 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b6dba-76b8-46e0-b820-5d5b549e8237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.307383 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b6dba-76b8-46e0-b820-5d5b549e8237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.308142 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b6dba-76b8-46e0-b820-5d5b549e8237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.309749 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b6dba-76b8-46e0-b820-5d5b549e8237 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.321813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1355/2723] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:14.329906 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb9268-d52f-46d1-9d8f-e1d037b645c8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.341741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb9268-d52f-46d1-9d8f-e1d037b645c8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.347144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022e4b-4292-442c-bbc7-51217441f319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.348184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022e4b-4292-442c-bbc7-51217441f319 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.348460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022e4b-4292-442c-bbc7-51217441f319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.348734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022e4b-4292-442c-bbc7-51217441f319 admin admin] PATH_INFO: `/v3/projects/93cdcc84d8ea40ada385c13ede70fcf6/users/e6e2261f21a04a6c9fd45088727945c4/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.366543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f022e4b-4292-442c-bbc7-51217441f319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93cdcc84d8ea40ada385c13ede70fcf6, user_id=e6e2261f21a04a6c9fd45088727945c4, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.369323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f022e4b-4292-442c-bbc7-51217441f319 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.385556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1369/2724] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:14 2022] PUT /identity/v3/projects/93cdcc84d8ea40ada385c13ede70fcf6/users/e6e2261f21a04a6c9fd45088727945c4/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:14.396493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee3431-f67d-42f5-adfc-f37dbba6be7e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.396779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee3431-f67d-42f5-adfc-f37dbba6be7e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.397054 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee3431-f67d-42f5-adfc-f37dbba6be7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.428138 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-06ee3431-f67d-42f5-adfc-f37dbba6be7e None None] MFA Rules not processed for user `e6e2261f21a04a6c9fd45088727945c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:14.429368 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-06ee3431-f67d-42f5-adfc-f37dbba6be7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:14.500937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1356/2725] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:36:14.506198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022e4b-4292-442c-bbc7-51217441f319 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.580632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022e4b-4292-442c-bbc7-51217441f319 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.585879 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60dbeb-156c-46aa-8e8d-dc32e82d72c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.586597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60dbeb-156c-46aa-8e8d-dc32e82d72c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.586889 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60dbeb-156c-46aa-8e8d-dc32e82d72c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.587158 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60dbeb-156c-46aa-8e8d-dc32e82d72c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.587855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60dbeb-156c-46aa-8e8d-dc32e82d72c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.589838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60dbeb-156c-46aa-8e8d-dc32e82d72c9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.617529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1370/2726] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:36:14 2022] POST /identity/v3/projects => generated 409 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:14.622125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee3431-f67d-42f5-adfc-f37dbba6be7e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.633494 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee3431-f67d-42f5-adfc-f37dbba6be7e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.661605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:14.696195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165a33d-d1c1-416f-b989-f69cbf69ee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.696880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165a33d-d1c1-416f-b989-f69cbf69ee19 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.697158 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165a33d-d1c1-416f-b989-f69cbf69ee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.697428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165a33d-d1c1-416f-b989-f69cbf69ee19 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.698131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165a33d-d1c1-416f-b989-f69cbf69ee19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.700097 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165a33d-d1c1-416f-b989-f69cbf69ee19 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.723482 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1357/2727] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:36:14 2022] POST /identity/v3/users => generated 311 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:14.728271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60dbeb-156c-46aa-8e8d-dc32e82d72c9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.739927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60dbeb-156c-46aa-8e8d-dc32e82d72c9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.745898 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188922c3-ff7d-48d0-9f9a-a0b23db68f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.747080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188922c3-ff7d-48d0-9f9a-a0b23db68f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.747540 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188922c3-ff7d-48d0-9f9a-a0b23db68f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.747830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188922c3-ff7d-48d0-9f9a-a0b23db68f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.748682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188922c3-ff7d-48d0-9f9a-a0b23db68f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.750793 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188922c3-ff7d-48d0-9f9a-a0b23db68f4d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.763886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1371/2728] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:14.768350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165a33d-d1c1-416f-b989-f69cbf69ee19 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.779795 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165a33d-d1c1-416f-b989-f69cbf69ee19 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.785452 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a952f5d-24fd-4f1a-8f5f-d3068f06a37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.786399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a952f5d-24fd-4f1a-8f5f-d3068f06a37f admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.786858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a952f5d-24fd-4f1a-8f5f-d3068f06a37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.787311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a952f5d-24fd-4f1a-8f5f-d3068f06a37f admin admin] PATH_INFO: `/v3/projects/7495199f48c242908784d709e3f27b4b/users/a05764ca50664a8f9a3efc1d63a298e8/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.806442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a952f5d-24fd-4f1a-8f5f-d3068f06a37f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7495199f48c242908784d709e3f27b4b, user_id=a05764ca50664a8f9a3efc1d63a298e8, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.809153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a952f5d-24fd-4f1a-8f5f-d3068f06a37f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.828012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1358/2729] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:14 2022] PUT /identity/v3/projects/7495199f48c242908784d709e3f27b4b/users/a05764ca50664a8f9a3efc1d63a298e8/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:14.832579 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188922c3-ff7d-48d0-9f9a-a0b23db68f4d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.843688 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188922c3-ff7d-48d0-9f9a-a0b23db68f4d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.907377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858677a-29e8-41fa-9009-faca21c09556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.908054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858677a-29e8-41fa-9009-faca21c09556 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.908333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858677a-29e8-41fa-9009-faca21c09556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.908608 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858677a-29e8-41fa-9009-faca21c09556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.909318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858677a-29e8-41fa-9009-faca21c09556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.911227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858677a-29e8-41fa-9009-faca21c09556 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.922927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1372/2730] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:14.927652 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a952f5d-24fd-4f1a-8f5f-d3068f06a37f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:14.938740 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a952f5d-24fd-4f1a-8f5f-d3068f06a37f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:14.944124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe2151-bb56-4591-adb7-9310bbc61963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:14.945068 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe2151-bb56-4591-adb7-9310bbc61963 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.945387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe2151-bb56-4591-adb7-9310bbc61963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.945784 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe2151-bb56-4591-adb7-9310bbc61963 admin admin] PATH_INFO: `/v3/projects/7495199f48c242908784d709e3f27b4b/users/a05764ca50664a8f9a3efc1d63a298e8/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:14.947961 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe2151-bb56-4591-adb7-9310bbc61963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7495199f48c242908784d709e3f27b4b, user_id=a05764ca50664a8f9a3efc1d63a298e8, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:14.950767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe2151-bb56-4591-adb7-9310bbc61963 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:14.962471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1359/2731] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:14 2022] PUT /identity/v3/projects/7495199f48c242908784d709e3f27b4b/users/a05764ca50664a8f9a3efc1d63a298e8/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:14.975047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9187eea-fbfc-4590-9562-e6d053a6c3f5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:14.975483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9187eea-fbfc-4590-9562-e6d053a6c3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:14.976032 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9187eea-fbfc-4590-9562-e6d053a6c3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:15.016659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a9187eea-fbfc-4590-9562-e6d053a6c3f5 None None] MFA Rules not processed for user `a05764ca50664a8f9a3efc1d63a298e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:15.017900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a9187eea-fbfc-4590-9562-e6d053a6c3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:15.056223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:15.098903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1373/2732] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:15.110038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe2151-bb56-4591-adb7-9310bbc61963 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:15.112129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4afe2151-bb56-4591-adb7-9310bbc61963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:15.173149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe2151-bb56-4591-adb7-9310bbc61963 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:15.177135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a127429-5fb7-46ca-8f1b-a1d57d54088a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:15.177881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a127429-5fb7-46ca-8f1b-a1d57d54088a service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:15.178168 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a127429-5fb7-46ca-8f1b-a1d57d54088a service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:15.178444 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a127429-5fb7-46ca-8f1b-a1d57d54088a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:15.188764 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a127429-5fb7-46ca-8f1b-a1d57d54088a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:15.190199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a127429-5fb7-46ca-8f1b-a1d57d54088a service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:15.200666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1360/2733] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:15.361385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9187eea-fbfc-4590-9562-e6d053a6c3f5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:15.371642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9187eea-fbfc-4590-9562-e6d053a6c3f5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:15.375436 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c3a83-7052-4f8c-856a-737e178d3417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:15.376069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c3a83-7052-4f8c-856a-737e178d3417 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:15.376352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c3a83-7052-4f8c-856a-737e178d3417 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:15.376629 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c3a83-7052-4f8c-856a-737e178d3417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:15.442405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c3a83-7052-4f8c-856a-737e178d3417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:15.444645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c3a83-7052-4f8c-856a-737e178d3417 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:15.454066 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1374/2734] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:23.908523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a127429-5fb7-46ca-8f1b-a1d57d54088a service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:23.978659 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a127429-5fb7-46ca-8f1b-a1d57d54088a service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:23.983726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:23.984662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:23.984920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:23.985193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] PATH_INFO: `/v3/users/e6e2261f21a04a6c9fd45088727945c4` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:23.986480 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e2261f21a04a6c9fd45088727945c4)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:23.988507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:24.015486 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:24.047065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e2261f21a04a6c9fd45088727945c4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.059980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e2261f21a04a6c9fd45088727945c4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.060733 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e2261f21a04a6c9fd45088727945c4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.061342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e2261f21a04a6c9fd45088727945c4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.069747 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e2261f21a04a6c9fd45088727945c4'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.085104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1361/2735] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:23 2022] DELETE /identity/v3/users/e6e2261f21a04a6c9fd45088727945c4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:24.090334 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a642aa-1e0e-4126-8495-7ae9095904ec None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:24.090880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a642aa-1e0e-4126-8495-7ae9095904ec None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:24.090967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a642aa-1e0e-4126-8495-7ae9095904ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:24.110538 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-70a642aa-1e0e-4126-8495-7ae9095904ec None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:24.111895 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-70a642aa-1e0e-4126-8495-7ae9095904ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:24.176240 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1375/2736] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:24.186824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:24.188316 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:24.257843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c383939-66d1-491f-87a3-0dd59215d87b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:24.261809 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574e2ac-2c20-4306-b820-3ea39365d1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:24.262520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574e2ac-2c20-4306-b820-3ea39365d1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:24.262883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574e2ac-2c20-4306-b820-3ea39365d1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:24.263199 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574e2ac-2c20-4306-b820-3ea39365d1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:24.273793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574e2ac-2c20-4306-b820-3ea39365d1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:24.275266 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574e2ac-2c20-4306-b820-3ea39365d1c7 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:24.287667 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1362/2737] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:24.576051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a642aa-1e0e-4126-8495-7ae9095904ec None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:24.577523 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-70a642aa-1e0e-4126-8495-7ae9095904ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:24.591154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a642aa-1e0e-4126-8495-7ae9095904ec None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:24.598367 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:24.599298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:24.599583 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:24.599859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] PATH_INFO: `/v3/projects/93cdcc84d8ea40ada385c13ede70fcf6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:24.601483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93cdcc84d8ea40ada385c13ede70fcf6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:24.603905 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:24.661485 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93cdcc84d8ea40ada385c13ede70fcf6', 'request_id': 'req-68d116e3-f030-4692-9b84-976c95dd8d76'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.670837 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93cdcc84d8ea40ada385c13ede70fcf6', 'request_id': 'req-68d116e3-f030-4692-9b84-976c95dd8d76'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.676881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:24.678872 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93cdcc84d8ea40ada385c13ede70fcf6', 'request_id': 'req-68d116e3-f030-4692-9b84-976c95dd8d76'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.693320 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] The token cache is being invalidate because project 93cdcc84d8ea40ada385c13ede70fcf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:24.693709 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.694591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.696197 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1376/2738] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:24 2022] DELETE /identity/v3/projects/93cdcc84d8ea40ada385c13ede70fcf6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:24.700258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574e2ac-2c20-4306-b820-3ea39365d1c7 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:24.710594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574e2ac-2c20-4306-b820-3ea39365d1c7 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:24.775390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd312a-8575-49ea-af02-23b755fb49f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:24.776322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd312a-8575-49ea-af02-23b755fb49f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:24.776613 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd312a-8575-49ea-af02-23b755fb49f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:24.776875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd312a-8575-49ea-af02-23b755fb49f4 admin admin] PATH_INFO: `/v3/users/a05764ca50664a8f9a3efc1d63a298e8` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:24.778250 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd312a-8575-49ea-af02-23b755fb49f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a05764ca50664a8f9a3efc1d63a298e8)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:24.780232 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd312a-8575-49ea-af02-23b755fb49f4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:24.832775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:24.834545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c4fd312a-8575-49ea-af02-23b755fb49f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a05764ca50664a8f9a3efc1d63a298e8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.848320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c4fd312a-8575-49ea-af02-23b755fb49f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a05764ca50664a8f9a3efc1d63a298e8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.849584 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c4fd312a-8575-49ea-af02-23b755fb49f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a05764ca50664a8f9a3efc1d63a298e8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.850389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c4fd312a-8575-49ea-af02-23b755fb49f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a05764ca50664a8f9a3efc1d63a298e8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.858597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c4fd312a-8575-49ea-af02-23b755fb49f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a05764ca50664a8f9a3efc1d63a298e8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:24.879822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1363/2739] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:24 2022] DELETE /identity/v3/users/a05764ca50664a8f9a3efc1d63a298e8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:25.217040 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:25.218668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:25.243625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d116e3-f030-4692-9b84-976c95dd8d76 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:25.314647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:25.315546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:25.315831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:25.316214 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] PATH_INFO: `/v3/projects/7495199f48c242908784d709e3f27b4b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:25.317711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7495199f48c242908784d709e3f27b4b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:25.319718 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:25.370162 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7495199f48c242908784d709e3f27b4b', 'request_id': 'req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:25.378915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7495199f48c242908784d709e3f27b4b', 'request_id': 'req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:25.385023 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7495199f48c242908784d709e3f27b4b', 'request_id': 'req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:25.404337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] The token cache is being invalidate because project 7495199f48c242908784d709e3f27b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:25.404337 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:25.405128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-dea06f04-c5a9-4040-90eb-4ef8d3f33d8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:25.407014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1377/2740] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:25 2022] DELETE /identity/v3/projects/7495199f48c242908784d709e3f27b4b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:36:25.412237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96c709-02bd-4fd3-ab94-789bb4894829 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:25.412742 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96c709-02bd-4fd3-ab94-789bb4894829 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:25.412833 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96c709-02bd-4fd3-ab94-789bb4894829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:25.433209 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-0b96c709-02bd-4fd3-ab94-789bb4894829 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:25.434421 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-0b96c709-02bd-4fd3-ab94-789bb4894829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:25.499615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1364/2741] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:25.507770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20c3e7-b28d-4041-a412-1ad3b583c1cc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:25.508064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20c3e7-b28d-4041-a412-1ad3b583c1cc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:25.508345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20c3e7-b28d-4041-a412-1ad3b583c1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:25.528283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4a20c3e7-b28d-4041-a412-1ad3b583c1cc None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:25.529515 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4a20c3e7-b28d-4041-a412-1ad3b583c1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:25.538051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1378/2742] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:25.543625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96c709-02bd-4fd3-ab94-789bb4894829 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:25.555137 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96c709-02bd-4fd3-ab94-789bb4894829 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:25.561291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d84171-d989-42a2-b40a-43d6f3b0c569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:25.561961 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84171-d989-42a2-b40a-43d6f3b0c569 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:25.562255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84171-d989-42a2-b40a-43d6f3b0c569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:25.562531 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84171-d989-42a2-b40a-43d6f3b0c569 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:25.563313 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d84171-d989-42a2-b40a-43d6f3b0c569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:36:25.563622 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d84171-d989-42a2-b40a-43d6f3b0c569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:25.565237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d84171-d989-42a2-b40a-43d6f3b0c569 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:25.579377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1365/2743] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:36:25.584543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20c3e7-b28d-4041-a412-1ad3b583c1cc None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:25.594489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20c3e7-b28d-4041-a412-1ad3b583c1cc None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:25.600212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f3531-f68c-4f9b-9280-3d4a1adea4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:25.600900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f3531-f68c-4f9b-9280-3d4a1adea4cd admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:25.601183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f3531-f68c-4f9b-9280-3d4a1adea4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:25.601459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f3531-f68c-4f9b-9280-3d4a1adea4cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:25.602159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f3531-f68c-4f9b-9280-3d4a1adea4cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:25.604165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f3531-f68c-4f9b-9280-3d4a1adea4cd admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:25.636854 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1379/2744] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:36:25 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:25.642499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d84171-d989-42a2-b40a-43d6f3b0c569 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:25.653999 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d84171-d989-42a2-b40a-43d6f3b0c569 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:25.723745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3dcf61-2a66-4080-a83e-6c15c8c7615c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:25.724769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dcf61-2a66-4080-a83e-6c15c8c7615c admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:25.724769 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dcf61-2a66-4080-a83e-6c15c8c7615c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:25.725067 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dcf61-2a66-4080-a83e-6c15c8c7615c admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:25.725671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3dcf61-2a66-4080-a83e-6c15c8c7615c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:25.727636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3dcf61-2a66-4080-a83e-6c15c8c7615c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:25.750928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1366/2745] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:36:25 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:25.757374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f3531-f68c-4f9b-9280-3d4a1adea4cd admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:25.773131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f3531-f68c-4f9b-9280-3d4a1adea4cd admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:25.779398 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec24974-6037-46a3-bd49-0c7b5aaee91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:25.780225 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec24974-6037-46a3-bd49-0c7b5aaee91c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:25.780499 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec24974-6037-46a3-bd49-0c7b5aaee91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:25.780766 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec24974-6037-46a3-bd49-0c7b5aaee91c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:25.781503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec24974-6037-46a3-bd49-0c7b5aaee91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:25.783169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec24974-6037-46a3-bd49-0c7b5aaee91c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:25.795283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1380/2746] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:25.799882 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3dcf61-2a66-4080-a83e-6c15c8c7615c admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:25.812272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3dcf61-2a66-4080-a83e-6c15c8c7615c admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:25.818792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99448df-3617-4ce7-8c8b-b1b1d2f8150a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:25.819528 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99448df-3617-4ce7-8c8b-b1b1d2f8150a admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:25.819806 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99448df-3617-4ce7-8c8b-b1b1d2f8150a admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:25.820076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99448df-3617-4ce7-8c8b-b1b1d2f8150a admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:25.820819 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99448df-3617-4ce7-8c8b-b1b1d2f8150a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:25.822474 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99448df-3617-4ce7-8c8b-b1b1d2f8150a admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:25.835610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1367/2747] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:25.840798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec24974-6037-46a3-bd49-0c7b5aaee91c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:25.852305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec24974-6037-46a3-bd49-0c7b5aaee91c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:25.858095 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c5fec-8e80-4261-ba4c-4a0b4c37ba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:25.859060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c5fec-8e80-4261-ba4c-4a0b4c37ba51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:25.859597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c5fec-8e80-4261-ba4c-4a0b4c37ba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:25.859881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c5fec-8e80-4261-ba4c-4a0b4c37ba51 admin admin] PATH_INFO: `/v3/projects/c8e5cda9996447ea9b9c640012eea445/users/db595022d0224a449d82519f6a24066d/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:25.878185 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c5fec-8e80-4261-ba4c-4a0b4c37ba51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e5cda9996447ea9b9c640012eea445, user_id=db595022d0224a449d82519f6a24066d, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:25.882285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c5fec-8e80-4261-ba4c-4a0b4c37ba51 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:25.900937 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1381/2748] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:25 2022] PUT /identity/v3/projects/c8e5cda9996447ea9b9c640012eea445/users/db595022d0224a449d82519f6a24066d/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:25.913453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124aa06-d794-4b15-95bd-8538e02ec5b4 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:25.913818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124aa06-d794-4b15-95bd-8538e02ec5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:25.914088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124aa06-d794-4b15-95bd-8538e02ec5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:25.944509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-d124aa06-d794-4b15-95bd-8538e02ec5b4 None None] MFA Rules not processed for user `db595022d0224a449d82519f6a24066d`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:25.945751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-d124aa06-d794-4b15-95bd-8538e02ec5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:25.963523 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:26.013735 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1368/2749] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:26.018434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c5fec-8e80-4261-ba4c-4a0b4c37ba51 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:26.041532 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:26.093133 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c5fec-8e80-4261-ba4c-4a0b4c37ba51 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:26.097990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e8010-a1a7-40a1-82c7-aee7f706a1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:26.098690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e8010-a1a7-40a1-82c7-aee7f706a1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:26.098991 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e8010-a1a7-40a1-82c7-aee7f706a1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:26.099264 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e8010-a1a7-40a1-82c7-aee7f706a1e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:26.100018 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e8010-a1a7-40a1-82c7-aee7f706a1e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:26.101957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e8010-a1a7-40a1-82c7-aee7f706a1e8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:26.130049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1382/2750] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:36:26 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:26.134462 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124aa06-d794-4b15-95bd-8538e02ec5b4 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:26.146141 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124aa06-d794-4b15-95bd-8538e02ec5b4 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:26.206813 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae73d0-becd-481c-bc26-0289f5271dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:26.207541 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae73d0-becd-481c-bc26-0289f5271dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:26.207820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae73d0-becd-481c-bc26-0289f5271dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:26.208096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae73d0-becd-481c-bc26-0289f5271dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:26.208800 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae73d0-becd-481c-bc26-0289f5271dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:26.210803 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae73d0-becd-481c-bc26-0289f5271dc7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:26.231663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1369/2751] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:36:26 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:36:26.236132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e8010-a1a7-40a1-82c7-aee7f706a1e8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:26.247774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e8010-a1a7-40a1-82c7-aee7f706a1e8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:26.253804 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2cc7e-d35f-4647-a442-261d2f90dd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:26.254498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2cc7e-d35f-4647-a442-261d2f90dd87 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:26.254813 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2cc7e-d35f-4647-a442-261d2f90dd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:26.255087 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2cc7e-d35f-4647-a442-261d2f90dd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:26.255827 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2cc7e-d35f-4647-a442-261d2f90dd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:26.257469 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2cc7e-d35f-4647-a442-261d2f90dd87 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:26.271014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1383/2752] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:26.275377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae73d0-becd-481c-bc26-0289f5271dc7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:26.286842 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae73d0-becd-481c-bc26-0289f5271dc7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:26.292253 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30dc1d-b3c3-44cc-8edd-5a3428958919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:26.293153 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dc1d-b3c3-44cc-8edd-5a3428958919 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:26.293430 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dc1d-b3c3-44cc-8edd-5a3428958919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:26.293699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dc1d-b3c3-44cc-8edd-5a3428958919 admin admin] PATH_INFO: `/v3/projects/c7a7ae32b45e4fb591dc2431a3820dda/users/34134185c1e84ce6bbd20ebeaab45bf8/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:26.312427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30dc1d-b3c3-44cc-8edd-5a3428958919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a7ae32b45e4fb591dc2431a3820dda, user_id=34134185c1e84ce6bbd20ebeaab45bf8, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:26.315151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30dc1d-b3c3-44cc-8edd-5a3428958919 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:26.329763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1370/2753] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:26 2022] PUT /identity/v3/projects/c7a7ae32b45e4fb591dc2431a3820dda/users/34134185c1e84ce6bbd20ebeaab45bf8/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:26.334052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2cc7e-d35f-4647-a442-261d2f90dd87 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:26.344395 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2cc7e-d35f-4647-a442-261d2f90dd87 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:26.404276 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f573f-894c-43e1-8810-afc95761fd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:26.404958 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f573f-894c-43e1-8810-afc95761fd5e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:26.405239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f573f-894c-43e1-8810-afc95761fd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:26.405513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f573f-894c-43e1-8810-afc95761fd5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:26.406251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f573f-894c-43e1-8810-afc95761fd5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:26.407934 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f573f-894c-43e1-8810-afc95761fd5e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:26.418603 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1384/2754] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:26.423299 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30dc1d-b3c3-44cc-8edd-5a3428958919 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:26.436233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30dc1d-b3c3-44cc-8edd-5a3428958919 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:26.442658 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06e73d-7f26-4124-aa9c-6adf1d5eba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:26.443693 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06e73d-7f26-4124-aa9c-6adf1d5eba8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:26.443981 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06e73d-7f26-4124-aa9c-6adf1d5eba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:26.444252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06e73d-7f26-4124-aa9c-6adf1d5eba8f admin admin] PATH_INFO: `/v3/projects/c7a7ae32b45e4fb591dc2431a3820dda/users/34134185c1e84ce6bbd20ebeaab45bf8/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:26.446350 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06e73d-7f26-4124-aa9c-6adf1d5eba8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a7ae32b45e4fb591dc2431a3820dda, user_id=34134185c1e84ce6bbd20ebeaab45bf8, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:26.449059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06e73d-7f26-4124-aa9c-6adf1d5eba8f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:26.460716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1371/2755] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:26 2022] PUT /identity/v3/projects/c7a7ae32b45e4fb591dc2431a3820dda/users/34134185c1e84ce6bbd20ebeaab45bf8/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:26.471682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6bdfb-740a-4bac-ba15-6b0aaae03577 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:26.471974 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6bdfb-740a-4bac-ba15-6b0aaae03577 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:26.472417 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6bdfb-740a-4bac-ba15-6b0aaae03577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:26.501094 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3bb6bdfb-740a-4bac-ba15-6b0aaae03577 None None] MFA Rules not processed for user `34134185c1e84ce6bbd20ebeaab45bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:26.502336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3bb6bdfb-740a-4bac-ba15-6b0aaae03577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:26.576084 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1385/2756] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:26.587276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06e73d-7f26-4124-aa9c-6adf1d5eba8f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:26.589331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ba06e73d-7f26-4124-aa9c-6adf1d5eba8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:26.650884 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06e73d-7f26-4124-aa9c-6adf1d5eba8f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:26.654948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5323fa3-744f-4178-b321-ad216c360007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:26.655610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5323fa3-744f-4178-b321-ad216c360007 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:26.655887 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5323fa3-744f-4178-b321-ad216c360007 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:26.656163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5323fa3-744f-4178-b321-ad216c360007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:26.666511 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5323fa3-744f-4178-b321-ad216c360007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:26.667986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5323fa3-744f-4178-b321-ad216c360007 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:26.678093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1372/2757] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:27.095237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6bdfb-740a-4bac-ba15-6b0aaae03577 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:27.109607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:27.161213 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6bdfb-740a-4bac-ba15-6b0aaae03577 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:27.166071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:27.166987 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:27.167283 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:27.167609 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] PATH_INFO: `/v3/users/db595022d0224a449d82519f6a24066d` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:27.168990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db595022d0224a449d82519f6a24066d)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:27.170990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:27.218016 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db595022d0224a449d82519f6a24066d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.229757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db595022d0224a449d82519f6a24066d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.230504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db595022d0224a449d82519f6a24066d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.237211 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db595022d0224a449d82519f6a24066d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.237909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db595022d0224a449d82519f6a24066d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.252094 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1386/2758] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:27 2022] DELETE /identity/v3/users/db595022d0224a449d82519f6a24066d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:27.256935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61773c-4055-451b-a255-e5747f005c3b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:27.257298 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61773c-4055-451b-a255-e5747f005c3b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:27.257585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61773c-4055-451b-a255-e5747f005c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:27.276409 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-bc61773c-4055-451b-a255-e5747f005c3b None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:27.277626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-bc61773c-4055-451b-a255-e5747f005c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:27.286196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:27.343556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1373/2759] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:27.353864 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:27.355596 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:27.416913 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5683ca-a412-4074-978b-8763be227c28 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:27.421413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fcfe2-c514-4575-b16d-288900194da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:27.422078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fcfe2-c514-4575-b16d-288900194da1 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:27.422368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fcfe2-c514-4575-b16d-288900194da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:27.422661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fcfe2-c514-4575-b16d-288900194da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:27.433383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fcfe2-c514-4575-b16d-288900194da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:27.434866 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fcfe2-c514-4575-b16d-288900194da1 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:27.444653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1387/2760] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:27.724666 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61773c-4055-451b-a255-e5747f005c3b None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:27.726167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-bc61773c-4055-451b-a255-e5747f005c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:27.741338 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61773c-4055-451b-a255-e5747f005c3b None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:27.747674 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:27.748656 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:27.748948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:27.749227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] PATH_INFO: `/v3/projects/c8e5cda9996447ea9b9c640012eea445` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:27.750497 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e5cda9996447ea9b9c640012eea445)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:27.752468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:27.808203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e5cda9996447ea9b9c640012eea445', 'request_id': 'req-f0600737-62fc-4aa4-9322-ae6925c8d32e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.817245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e5cda9996447ea9b9c640012eea445', 'request_id': 'req-f0600737-62fc-4aa4-9322-ae6925c8d32e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.823025 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e5cda9996447ea9b9c640012eea445', 'request_id': 'req-f0600737-62fc-4aa4-9322-ae6925c8d32e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.835012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] The token cache is being invalidate because project c8e5cda9996447ea9b9c640012eea445 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:27.835302 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.835985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.837685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1374/2761] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:27 2022] DELETE /identity/v3/projects/c8e5cda9996447ea9b9c640012eea445 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:27.841634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fcfe2-c514-4575-b16d-288900194da1 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:27.852074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fcfe2-c514-4575-b16d-288900194da1 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:27.887114 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:27.915487 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c9ddf-5cb1-4c3a-a3b5-b61e055478fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:27.916380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c9ddf-5cb1-4c3a-a3b5-b61e055478fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:27.916664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c9ddf-5cb1-4c3a-a3b5-b61e055478fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:27.916966 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c9ddf-5cb1-4c3a-a3b5-b61e055478fc admin admin] PATH_INFO: `/v3/users/34134185c1e84ce6bbd20ebeaab45bf8` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:27.918174 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c9ddf-5cb1-4c3a-a3b5-b61e055478fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=34134185c1e84ce6bbd20ebeaab45bf8)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:27.920198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c9ddf-5cb1-4c3a-a3b5-b61e055478fc admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:27.969421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c92c9ddf-5cb1-4c3a-a3b5-b61e055478fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34134185c1e84ce6bbd20ebeaab45bf8'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.982080 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c92c9ddf-5cb1-4c3a-a3b5-b61e055478fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34134185c1e84ce6bbd20ebeaab45bf8'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.982844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c92c9ddf-5cb1-4c3a-a3b5-b61e055478fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34134185c1e84ce6bbd20ebeaab45bf8'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.990271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c92c9ddf-5cb1-4c3a-a3b5-b61e055478fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34134185c1e84ce6bbd20ebeaab45bf8'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:27.990939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c92c9ddf-5cb1-4c3a-a3b5-b61e055478fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34134185c1e84ce6bbd20ebeaab45bf8'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:28.005154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1388/2762] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:27 2022] DELETE /identity/v3/users/34134185c1e84ce6bbd20ebeaab45bf8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:28.296005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:28.297590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:28.312454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0600737-62fc-4aa4-9322-ae6925c8d32e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:28.375629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:28.376428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:28.376712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:28.376984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] PATH_INFO: `/v3/projects/c7a7ae32b45e4fb591dc2431a3820dda` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:28.378194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7a7ae32b45e4fb591dc2431a3820dda)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:28.380434 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:28.424712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:28.434346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7a7ae32b45e4fb591dc2431a3820dda', 'request_id': 'req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:28.442467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7a7ae32b45e4fb591dc2431a3820dda', 'request_id': 'req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:28.448428 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7a7ae32b45e4fb591dc2431a3820dda', 'request_id': 'req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:28.460213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] The token cache is being invalidate because project c7a7ae32b45e4fb591dc2431a3820dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:28.460501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:28.461236 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-2b9eb1f5-e1fc-4464-9c49-d09a408c003c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:28.463138 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1375/2763] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:28 2022] DELETE /identity/v3/projects/c7a7ae32b45e4fb591dc2431a3820dda => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:28.468271 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a215c-1cec-4628-af94-81b99fa6cec9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:28.468777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a215c-1cec-4628-af94-81b99fa6cec9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:28.468777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a215c-1cec-4628-af94-81b99fa6cec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:28.489577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1b0a215c-1cec-4628-af94-81b99fa6cec9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:28.490823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1b0a215c-1cec-4628-af94-81b99fa6cec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:28.559552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1389/2764] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:28.567801 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111fe06-1676-42ff-9fd4-ef4e45f57fd4 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:28.568071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111fe06-1676-42ff-9fd4-ef4e45f57fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:28.568361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111fe06-1676-42ff-9fd4-ef4e45f57fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:28.587815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6111fe06-1676-42ff-9fd4-ef4e45f57fd4 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:28.589163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6111fe06-1676-42ff-9fd4-ef4e45f57fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:28.598154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1376/2765] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:28.602752 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a215c-1cec-4628-af94-81b99fa6cec9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:28.613305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a215c-1cec-4628-af94-81b99fa6cec9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:28.618846 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ed2b0-7ad3-4a7d-8b54-ceaeae0debbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:28.619504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ed2b0-7ad3-4a7d-8b54-ceaeae0debbe admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:28.619781 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ed2b0-7ad3-4a7d-8b54-ceaeae0debbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:28.620050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ed2b0-7ad3-4a7d-8b54-ceaeae0debbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:28.620777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ed2b0-7ad3-4a7d-8b54-ceaeae0debbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:36:28.621082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ed2b0-7ad3-4a7d-8b54-ceaeae0debbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:28.622842 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ed2b0-7ad3-4a7d-8b54-ceaeae0debbe admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:28.636106 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1390/2766] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:36:28.640711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111fe06-1676-42ff-9fd4-ef4e45f57fd4 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:28.650935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111fe06-1676-42ff-9fd4-ef4e45f57fd4 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:28.656632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e4a43-2bb3-4102-bdc5-63e023174a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:28.657451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e4a43-2bb3-4102-bdc5-63e023174a93 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:28.657834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e4a43-2bb3-4102-bdc5-63e023174a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:28.658152 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e4a43-2bb3-4102-bdc5-63e023174a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:28.658955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e4a43-2bb3-4102-bdc5-63e023174a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:28.660937 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e4a43-2bb3-4102-bdc5-63e023174a93 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:28.691041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1377/2767] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:36:28 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:28.695606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ed2b0-7ad3-4a7d-8b54-ceaeae0debbe admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:28.706131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ed2b0-7ad3-4a7d-8b54-ceaeae0debbe admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:28.749922 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:28.768710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a089f-7000-4e18-861e-4236e090773d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:28.769583 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a089f-7000-4e18-861e-4236e090773d admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:28.769868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a089f-7000-4e18-861e-4236e090773d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:28.770149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a089f-7000-4e18-861e-4236e090773d admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:28.770891 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a089f-7000-4e18-861e-4236e090773d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:28.772845 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a089f-7000-4e18-861e-4236e090773d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:28.794537 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1391/2768] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:36:28 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:28.797712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e4a43-2bb3-4102-bdc5-63e023174a93 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:28.809433 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e4a43-2bb3-4102-bdc5-63e023174a93 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:28.814810 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e0093-6f5a-41a5-9e45-186a38defbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:28.815492 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e0093-6f5a-41a5-9e45-186a38defbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:28.815774 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e0093-6f5a-41a5-9e45-186a38defbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:28.816069 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e0093-6f5a-41a5-9e45-186a38defbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:28.816798 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e0093-6f5a-41a5-9e45-186a38defbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:28.818399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e0093-6f5a-41a5-9e45-186a38defbe9 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:28.830059 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1378/2769] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:28.834393 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a089f-7000-4e18-861e-4236e090773d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:28.846993 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a089f-7000-4e18-861e-4236e090773d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:28.852590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53209cdb-f838-4d26-ba2d-554e5dee8787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:28.853275 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53209cdb-f838-4d26-ba2d-554e5dee8787 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:28.853558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53209cdb-f838-4d26-ba2d-554e5dee8787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:28.853835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53209cdb-f838-4d26-ba2d-554e5dee8787 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:28.854590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53209cdb-f838-4d26-ba2d-554e5dee8787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:28.856221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53209cdb-f838-4d26-ba2d-554e5dee8787 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:28.867529 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1392/2770] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:28.871985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e0093-6f5a-41a5-9e45-186a38defbe9 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:28.891002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e0093-6f5a-41a5-9e45-186a38defbe9 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:28.896807 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fc224-a6b8-4935-b902-02d23233b4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:28.897713 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc224-a6b8-4935-b902-02d23233b4dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:28.897991 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc224-a6b8-4935-b902-02d23233b4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:28.898264 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc224-a6b8-4935-b902-02d23233b4dd admin admin] PATH_INFO: `/v3/projects/dc11e0db3a6e4ad1a8dfec2f8adddf53/users/2effa3ac42fe49318500ff6924690de5/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:28.914260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fc224-a6b8-4935-b902-02d23233b4dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc11e0db3a6e4ad1a8dfec2f8adddf53, user_id=2effa3ac42fe49318500ff6924690de5, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:28.916982 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fc224-a6b8-4935-b902-02d23233b4dd admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:28.931570 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1379/2771] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:28 2022] PUT /identity/v3/projects/dc11e0db3a6e4ad1a8dfec2f8adddf53/users/2effa3ac42fe49318500ff6924690de5/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:36:28.943058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263fb9a-7e8a-4c94-bf2c-b97a799453c1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:28.943348 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263fb9a-7e8a-4c94-bf2c-b97a799453c1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:28.943627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263fb9a-7e8a-4c94-bf2c-b97a799453c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:28.972577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f263fb9a-7e8a-4c94-bf2c-b97a799453c1 None None] MFA Rules not processed for user `2effa3ac42fe49318500ff6924690de5`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:28.973836 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f263fb9a-7e8a-4c94-bf2c-b97a799453c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:29.040440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1393/2772] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:29.045510 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fc224-a6b8-4935-b902-02d23233b4dd admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:29.112364 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fc224-a6b8-4935-b902-02d23233b4dd admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:29.117904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c6154-6621-4ca4-8bef-c9b42ad05266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:29.118605 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c6154-6621-4ca4-8bef-c9b42ad05266 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:29.118897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c6154-6621-4ca4-8bef-c9b42ad05266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:29.119173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c6154-6621-4ca4-8bef-c9b42ad05266 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:29.119869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c6154-6621-4ca4-8bef-c9b42ad05266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:29.121768 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c6154-6621-4ca4-8bef-c9b42ad05266 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:29.148555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1380/2773] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:36:29 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:29.152881 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263fb9a-7e8a-4c94-bf2c-b97a799453c1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:29.164146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263fb9a-7e8a-4c94-bf2c-b97a799453c1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:29.223630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff72fd-aa3b-451a-8902-ee52d8a9c990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:29.224313 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff72fd-aa3b-451a-8902-ee52d8a9c990 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:29.224592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff72fd-aa3b-451a-8902-ee52d8a9c990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:29.224871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff72fd-aa3b-451a-8902-ee52d8a9c990 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:29.225593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff72fd-aa3b-451a-8902-ee52d8a9c990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:29.227582 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff72fd-aa3b-451a-8902-ee52d8a9c990 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:29.249931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1394/2774] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:36:29 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:29.254985 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c6154-6621-4ca4-8bef-c9b42ad05266 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:29.265956 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c6154-6621-4ca4-8bef-c9b42ad05266 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:29.271499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b46385-c5c2-4ad9-af4e-dd40868aae5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:29.272183 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b46385-c5c2-4ad9-af4e-dd40868aae5d admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:29.272468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b46385-c5c2-4ad9-af4e-dd40868aae5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:29.272745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b46385-c5c2-4ad9-af4e-dd40868aae5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:29.273488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b46385-c5c2-4ad9-af4e-dd40868aae5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:29.275155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b46385-c5c2-4ad9-af4e-dd40868aae5d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:29.288186 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1381/2775] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:29.292643 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff72fd-aa3b-451a-8902-ee52d8a9c990 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:29.303621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff72fd-aa3b-451a-8902-ee52d8a9c990 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:29.309888 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9107316-d990-4682-9de9-d881418a76d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:29.310844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9107316-d990-4682-9de9-d881418a76d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:29.311127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9107316-d990-4682-9de9-d881418a76d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:29.311401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9107316-d990-4682-9de9-d881418a76d8 admin admin] PATH_INFO: `/v3/projects/68d23e1b9e494ecb938b2a76d98f63a1/users/95c10251e75143e882804d78742d4778/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:29.328209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9107316-d990-4682-9de9-d881418a76d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d23e1b9e494ecb938b2a76d98f63a1, user_id=95c10251e75143e882804d78742d4778, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:29.330949 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9107316-d990-4682-9de9-d881418a76d8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:29.346061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1395/2776] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:29 2022] PUT /identity/v3/projects/68d23e1b9e494ecb938b2a76d98f63a1/users/95c10251e75143e882804d78742d4778/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:29.350385 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b46385-c5c2-4ad9-af4e-dd40868aae5d admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:29.360625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b46385-c5c2-4ad9-af4e-dd40868aae5d admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:29.395460 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:29.420997 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a6c49-9044-4ba5-9643-8327acb9b036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:29.421600 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a6c49-9044-4ba5-9643-8327acb9b036 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:29.421877 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a6c49-9044-4ba5-9643-8327acb9b036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:29.422149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a6c49-9044-4ba5-9643-8327acb9b036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:29.422926 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a6c49-9044-4ba5-9643-8327acb9b036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:29.424533 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a6c49-9044-4ba5-9643-8327acb9b036 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:29.435237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1382/2777] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:29.439568 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9107316-d990-4682-9de9-d881418a76d8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:29.451124 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9107316-d990-4682-9de9-d881418a76d8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:29.457039 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1e400-ab85-4afe-b3e2-47282458f995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:29.457944 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1e400-ab85-4afe-b3e2-47282458f995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:29.458218 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1e400-ab85-4afe-b3e2-47282458f995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:29.458506 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1e400-ab85-4afe-b3e2-47282458f995 admin admin] PATH_INFO: `/v3/projects/68d23e1b9e494ecb938b2a76d98f63a1/users/95c10251e75143e882804d78742d4778/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:29.460731 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df1e400-ab85-4afe-b3e2-47282458f995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d23e1b9e494ecb938b2a76d98f63a1, user_id=95c10251e75143e882804d78742d4778, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:29.463475 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df1e400-ab85-4afe-b3e2-47282458f995 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:29.483815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1396/2778] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:29 2022] PUT /identity/v3/projects/68d23e1b9e494ecb938b2a76d98f63a1/users/95c10251e75143e882804d78742d4778/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:29.485650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9f1d5-587f-4dc4-81fe-afbc29fa48ee None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:29.485942 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9f1d5-587f-4dc4-81fe-afbc29fa48ee None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:29.486248 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9f1d5-587f-4dc4-81fe-afbc29fa48ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:29.514072 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-13a9f1d5-587f-4dc4-81fe-afbc29fa48ee None None] MFA Rules not processed for user `95c10251e75143e882804d78742d4778`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:29.515378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-13a9f1d5-587f-4dc4-81fe-afbc29fa48ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:29.588233 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1383/2779] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:29.599312 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1e400-ab85-4afe-b3e2-47282458f995 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:29.601043 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0df1e400-ab85-4afe-b3e2-47282458f995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:29.606659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:29.664125 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1e400-ab85-4afe-b3e2-47282458f995 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:29.668541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd46cff-79de-4d05-a03f-a61dabb3eeb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:29.669198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd46cff-79de-4d05-a03f-a61dabb3eeb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:29.669646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd46cff-79de-4d05-a03f-a61dabb3eeb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:29.669912 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd46cff-79de-4d05-a03f-a61dabb3eeb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:29.730208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd46cff-79de-4d05-a03f-a61dabb3eeb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:29.731994 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd46cff-79de-4d05-a03f-a61dabb3eeb5 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:29.741021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1397/2780] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:32.320869 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9f1d5-587f-4dc4-81fe-afbc29fa48ee None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:32.395917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9f1d5-587f-4dc4-81fe-afbc29fa48ee None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:32.401043 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:32.401946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:32.402221 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:32.402488 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] PATH_INFO: `/v3/users/2effa3ac42fe49318500ff6924690de5` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:32.403745 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2effa3ac42fe49318500ff6924690de5)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:32.405696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:32.471403 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2effa3ac42fe49318500ff6924690de5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:32.484354 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:32.487384 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2effa3ac42fe49318500ff6924690de5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:32.489215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2effa3ac42fe49318500ff6924690de5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:32.490637 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2effa3ac42fe49318500ff6924690de5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:32.499816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2effa3ac42fe49318500ff6924690de5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:32.517629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1384/2781] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:32 2022] DELETE /identity/v3/users/2effa3ac42fe49318500ff6924690de5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:32.522659 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d3d9-c0b8-4825-912e-e4dd8105becb None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:32.523142 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d3d9-c0b8-4825-912e-e4dd8105becb None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:32.523251 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d3d9-c0b8-4825-912e-e4dd8105becb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:32.543723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4d03d3d9-c0b8-4825-912e-e4dd8105becb None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:32.545210 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4d03d3d9-c0b8-4825-912e-e4dd8105becb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:32.614756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1398/2782] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:32.626110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:32.627844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:32.697907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e19307-7ade-4f9f-92cf-8e4bc8e55253 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:32.702288 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1879f9-1b10-4bb6-8283-291a01f4cffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:32.703021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1879f9-1b10-4bb6-8283-291a01f4cffb service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:32.703331 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1879f9-1b10-4bb6-8283-291a01f4cffb service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:32.703641 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1879f9-1b10-4bb6-8283-291a01f4cffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:32.714096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1879f9-1b10-4bb6-8283-291a01f4cffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:32.715634 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1879f9-1b10-4bb6-8283-291a01f4cffb service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:32.726099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1385/2783] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:33.044810 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03d3d9-c0b8-4825-912e-e4dd8105becb None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:33.044810 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4d03d3d9-c0b8-4825-912e-e4dd8105becb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:33.074202 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03d3d9-c0b8-4825-912e-e4dd8105becb None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:33.083544 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:33.084428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:33.084711 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:33.084981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] PATH_INFO: `/v3/projects/dc11e0db3a6e4ad1a8dfec2f8adddf53` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:33.086304 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc11e0db3a6e4ad1a8dfec2f8adddf53)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:33.088391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:33.142884 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc11e0db3a6e4ad1a8dfec2f8adddf53', 'request_id': 'req-a4a3023e-58e8-4431-9351-44cd21986881'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.146300 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:33.152868 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc11e0db3a6e4ad1a8dfec2f8adddf53', 'request_id': 'req-a4a3023e-58e8-4431-9351-44cd21986881'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.161697 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc11e0db3a6e4ad1a8dfec2f8adddf53', 'request_id': 'req-a4a3023e-58e8-4431-9351-44cd21986881'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.174981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] The token cache is being invalidate because project dc11e0db3a6e4ad1a8dfec2f8adddf53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:33.175322 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.176892 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.178902 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1399/2784] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:33 2022] DELETE /identity/v3/projects/dc11e0db3a6e4ad1a8dfec2f8adddf53 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:33.183586 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1879f9-1b10-4bb6-8283-291a01f4cffb service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:33.199154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1879f9-1b10-4bb6-8283-291a01f4cffb service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:33.268224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e5a9e-881b-423e-947c-0f545284fec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:33.268224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e5a9e-881b-423e-947c-0f545284fec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:33.268224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e5a9e-881b-423e-947c-0f545284fec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:33.268224 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e5a9e-881b-423e-947c-0f545284fec8 admin admin] PATH_INFO: `/v3/users/95c10251e75143e882804d78742d4778` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:33.268683 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e5a9e-881b-423e-947c-0f545284fec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c10251e75143e882804d78742d4778)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:33.268767 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e5a9e-881b-423e-947c-0f545284fec8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:33.327698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d74e5a9e-881b-423e-947c-0f545284fec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c10251e75143e882804d78742d4778'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.341988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d74e5a9e-881b-423e-947c-0f545284fec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c10251e75143e882804d78742d4778'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.342858 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d74e5a9e-881b-423e-947c-0f545284fec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c10251e75143e882804d78742d4778'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.343619 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d74e5a9e-881b-423e-947c-0f545284fec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c10251e75143e882804d78742d4778'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.351714 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-d74e5a9e-881b-423e-947c-0f545284fec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c10251e75143e882804d78742d4778'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.364415 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:33.371201 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1386/2785] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:33 2022] DELETE /identity/v3/users/95c10251e75143e882804d78742d4778 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:33.700241 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:33.701746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:33.719956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3023e-58e8-4431-9351-44cd21986881 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:33.795489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:33.798785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:33.798785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:33.798785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] PATH_INFO: `/v3/projects/68d23e1b9e494ecb938b2a76d98f63a1` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:33.798785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d23e1b9e494ecb938b2a76d98f63a1)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:33.800732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:33.866885 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d23e1b9e494ecb938b2a76d98f63a1', 'request_id': 'req-14c98489-dd53-487b-8234-06ee1eab01a2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.880298 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d23e1b9e494ecb938b2a76d98f63a1', 'request_id': 'req-14c98489-dd53-487b-8234-06ee1eab01a2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.886735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d23e1b9e494ecb938b2a76d98f63a1', 'request_id': 'req-14c98489-dd53-487b-8234-06ee1eab01a2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.899522 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] The token cache is being invalidate because project 68d23e1b9e494ecb938b2a76d98f63a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:33.899806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.900477 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-14c98489-dd53-487b-8234-06ee1eab01a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:33.902085 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1400/2786] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:33 2022] DELETE /identity/v3/projects/68d23e1b9e494ecb938b2a76d98f63a1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:33.907792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d250190-ae70-44a5-a19e-34023d715dfa None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:33.908104 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d250190-ae70-44a5-a19e-34023d715dfa None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:33.908385 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d250190-ae70-44a5-a19e-34023d715dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:33.928984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-7d250190-ae70-44a5-a19e-34023d715dfa None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:33.930410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7d250190-ae70-44a5-a19e-34023d715dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:34.008754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1387/2787] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:34.016798 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a931af-def5-4140-b4df-e5d21fa206ea None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:34.017216 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a931af-def5-4140-b4df-e5d21fa206ea None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:34.017492 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a931af-def5-4140-b4df-e5d21fa206ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:34.043404 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-36a931af-def5-4140-b4df-e5d21fa206ea None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:34.044667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-36a931af-def5-4140-b4df-e5d21fa206ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:34.054314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1401/2788] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:36:34.059389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d250190-ae70-44a5-a19e-34023d715dfa None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:34.073260 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d250190-ae70-44a5-a19e-34023d715dfa None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:34.080739 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8cd92-a4dc-45eb-93d6-5c7fe3c1344e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:34.081571 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8cd92-a4dc-45eb-93d6-5c7fe3c1344e admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:34.081940 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8cd92-a4dc-45eb-93d6-5c7fe3c1344e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:34.082454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8cd92-a4dc-45eb-93d6-5c7fe3c1344e admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:34.083296 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8cd92-a4dc-45eb-93d6-5c7fe3c1344e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:36:34.083615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8cd92-a4dc-45eb-93d6-5c7fe3c1344e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:34.085618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8cd92-a4dc-45eb-93d6-5c7fe3c1344e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:34.100163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1388/2789] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:36:34.105297 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a931af-def5-4140-b4df-e5d21fa206ea None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:34.115714 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a931af-def5-4140-b4df-e5d21fa206ea None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:34.120556 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08b20f-1d1e-46f1-a595-a47633270c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:34.121785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08b20f-1d1e-46f1-a595-a47633270c2d admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:34.121785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08b20f-1d1e-46f1-a595-a47633270c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:34.121785 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08b20f-1d1e-46f1-a595-a47633270c2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:34.122434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08b20f-1d1e-46f1-a595-a47633270c2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:34.124450 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08b20f-1d1e-46f1-a595-a47633270c2d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:34.166233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1402/2790] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:36:34 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:36:34.171181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8cd92-a4dc-45eb-93d6-5c7fe3c1344e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:34.185875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8cd92-a4dc-45eb-93d6-5c7fe3c1344e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:34.271962 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490175c-414d-48fd-af1d-966d739ee356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:34.272992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490175c-414d-48fd-af1d-966d739ee356 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:34.273125 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490175c-414d-48fd-af1d-966d739ee356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:34.273339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490175c-414d-48fd-af1d-966d739ee356 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:34.274319 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490175c-414d-48fd-af1d-966d739ee356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:34.277171 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490175c-414d-48fd-af1d-966d739ee356 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:34.307407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1389/2791] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:36:34 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:36:34.313504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08b20f-1d1e-46f1-a595-a47633270c2d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:34.325374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08b20f-1d1e-46f1-a595-a47633270c2d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:34.331119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d668a7-e55d-4c95-af00-dd8a635427f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:34.331794 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d668a7-e55d-4c95-af00-dd8a635427f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:34.332070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d668a7-e55d-4c95-af00-dd8a635427f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:34.332373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d668a7-e55d-4c95-af00-dd8a635427f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:34.333061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d668a7-e55d-4c95-af00-dd8a635427f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:34.334704 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d668a7-e55d-4c95-af00-dd8a635427f0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:34.348121 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1403/2792] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:34.352906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490175c-414d-48fd-af1d-966d739ee356 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:34.365781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490175c-414d-48fd-af1d-966d739ee356 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:34.371699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceea9fb-c3cb-480a-ab4c-a3f37b3543f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:34.372374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceea9fb-c3cb-480a-ab4c-a3f37b3543f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:34.372649 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceea9fb-c3cb-480a-ab4c-a3f37b3543f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:34.372920 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceea9fb-c3cb-480a-ab4c-a3f37b3543f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:34.373653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceea9fb-c3cb-480a-ab4c-a3f37b3543f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:34.375437 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceea9fb-c3cb-480a-ab4c-a3f37b3543f0 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:34.390547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1390/2793] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:36:34.397333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d668a7-e55d-4c95-af00-dd8a635427f0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:34.411576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d668a7-e55d-4c95-af00-dd8a635427f0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:34.417274 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6af1b1-00d2-40b6-96ac-4df1aebf2ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:34.418188 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6af1b1-00d2-40b6-96ac-4df1aebf2ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:34.418473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6af1b1-00d2-40b6-96ac-4df1aebf2ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:34.418770 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6af1b1-00d2-40b6-96ac-4df1aebf2ea6 admin admin] PATH_INFO: `/v3/projects/556b0713c53744e1a58fed0756ac3a13/users/79681dafdc8546c2b641e10026360fee/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:34.435677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6af1b1-00d2-40b6-96ac-4df1aebf2ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556b0713c53744e1a58fed0756ac3a13, user_id=79681dafdc8546c2b641e10026360fee, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:34.438366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6af1b1-00d2-40b6-96ac-4df1aebf2ea6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:34.453574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1404/2794] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:36:34 2022] PUT /identity/v3/projects/556b0713c53744e1a58fed0756ac3a13/users/79681dafdc8546c2b641e10026360fee/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:34.465839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378afae-cfc4-479f-bce6-3bf318ae379d None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:34.466148 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378afae-cfc4-479f-bce6-3bf318ae379d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:34.466453 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378afae-cfc4-479f-bce6-3bf318ae379d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:34.502220 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a378afae-cfc4-479f-bce6-3bf318ae379d None None] MFA Rules not processed for user `79681dafdc8546c2b641e10026360fee`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:34.504124 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a378afae-cfc4-479f-bce6-3bf318ae379d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:34.574761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:34.586456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1391/2795] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:34.599392 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6af1b1-00d2-40b6-96ac-4df1aebf2ea6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:34.602193 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fe6af1b1-00d2-40b6-96ac-4df1aebf2ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:34.616855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:34.680286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6af1b1-00d2-40b6-96ac-4df1aebf2ea6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:34.685552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f418-7db0-499a-826d-3fb6f1b9e964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:34.686282 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f418-7db0-499a-826d-3fb6f1b9e964 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:34.686726 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f418-7db0-499a-826d-3fb6f1b9e964 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:34.687100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f418-7db0-499a-826d-3fb6f1b9e964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:34.702402 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81f418-7db0-499a-826d-3fb6f1b9e964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:34.704333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81f418-7db0-499a-826d-3fb6f1b9e964 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:34.716327 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1405/2796] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:36:36.126726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b20bf-5866-4f7d-b7cf-5a7eebc74e6a None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:36.127486 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b20bf-5866-4f7d-b7cf-5a7eebc74e6a None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:36.127601 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b20bf-5866-4f7d-b7cf-5a7eebc74e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:36.150878 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6a7b20bf-5866-4f7d-b7cf-5a7eebc74e6a None None] MFA Rules not processed for user `79681dafdc8546c2b641e10026360fee`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:36.152117 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6a7b20bf-5866-4f7d-b7cf-5a7eebc74e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:36.159836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1392/2797] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:44.711410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=23 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:46.145273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:49.842628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f418-7db0-499a-826d-3fb6f1b9e964 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:49.844184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1b81f418-7db0-499a-826d-3fb6f1b9e964 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:49.980452 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f418-7db0-499a-826d-3fb6f1b9e964 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:49.992909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba660e4-01a6-409d-8e6b-e04dbcdea63e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:49.993577 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba660e4-01a6-409d-8e6b-e04dbcdea63e service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:49.993850 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba660e4-01a6-409d-8e6b-e04dbcdea63e service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:49.994199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba660e4-01a6-409d-8e6b-e04dbcdea63e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:49.996219 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-eba660e4-01a6-409d-8e6b-e04dbcdea63e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:50.013451 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba660e4-01a6-409d-8e6b-e04dbcdea63e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:50.016055 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba660e4-01a6-409d-8e6b-e04dbcdea63e service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:50.027771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1406/2798] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:51.446731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198014fb-5129-45c6-b141-7c399212a01b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:51.447306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198014fb-5129-45c6-b141-7c399212a01b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:51.447306 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198014fb-5129-45c6-b141-7c399212a01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:51.511103 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-198014fb-5129-45c6-b141-7c399212a01b None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:51.512311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-198014fb-5129-45c6-b141-7c399212a01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:51.581226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1393/2799] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:51.635149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba660e4-01a6-409d-8e6b-e04dbcdea63e service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:51.636999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-eba660e4-01a6-409d-8e6b-e04dbcdea63e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:51.734173 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba660e4-01a6-409d-8e6b-e04dbcdea63e service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:51.739486 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f8c57-c3d8-4e9a-9574-a137bb798407 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:51.740139 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f8c57-c3d8-4e9a-9574-a137bb798407 service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:51.740421 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f8c57-c3d8-4e9a-9574-a137bb798407 service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:51.740693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f8c57-c3d8-4e9a-9574-a137bb798407 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:51.750587 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f8c57-c3d8-4e9a-9574-a137bb798407 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:51.752208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f8c57-c3d8-4e9a-9574-a137bb798407 service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:51.760535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1407/2800] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:36:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:36:52.345149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bc678-1df1-4a9c-b280-35b479c90bbb None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:52.345561 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bc678-1df1-4a9c-b280-35b479c90bbb None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:52.346021 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bc678-1df1-4a9c-b280-35b479c90bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:52.367912 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-a06bc678-1df1-4a9c-b280-35b479c90bbb None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:52.369167 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a06bc678-1df1-4a9c-b280-35b479c90bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:52.378041 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1394/2801] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jun 29 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:52.391956 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f8c57-c3d8-4e9a-9574-a137bb798407 service swift] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:52.403799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f8c57-c3d8-4e9a-9574-a137bb798407 service swift] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:52.409632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f8215-d21d-491c-8945-9b1636496776 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:52.410291 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8215-d21d-491c-8945-9b1636496776 service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:52.410598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8215-d21d-491c-8945-9b1636496776 service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:52.410867 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8215-d21d-491c-8945-9b1636496776 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:52.420762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f8215-d21d-491c-8945-9b1636496776 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:52.422371 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f8215-d21d-491c-8945-9b1636496776 service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:52.432229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1408/2802] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:36:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:36:52.646423 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52ca6e-095f-4aeb-a7c8-48d6b158dd4e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:52.647005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52ca6e-095f-4aeb-a7c8-48d6b158dd4e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:52.647005 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52ca6e-095f-4aeb-a7c8-48d6b158dd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:52.667915 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-cb52ca6e-095f-4aeb-a7c8-48d6b158dd4e None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:52.669154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-cb52ca6e-095f-4aeb-a7c8-48d6b158dd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:52.677024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1395/2803] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:52.692112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f8215-d21d-491c-8945-9b1636496776 service swift] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:52.704479 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f8215-d21d-491c-8945-9b1636496776 service swift] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:52.710003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8e431-e8f5-43ec-bbe3-dbce30229a9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:52.710728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8e431-e8f5-43ec-bbe3-dbce30229a9e service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:52.711004 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8e431-e8f5-43ec-bbe3-dbce30229a9e service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:52.711278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8e431-e8f5-43ec-bbe3-dbce30229a9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:52.723317 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8e431-e8f5-43ec-bbe3-dbce30229a9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:52.724985 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8e431-e8f5-43ec-bbe3-dbce30229a9e service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:52.735155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1409/2804] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:36:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:36:53.101247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb77808-cec9-42a2-b1f1-99404f330b1c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:53.101539 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb77808-cec9-42a2-b1f1-99404f330b1c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:53.101821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb77808-cec9-42a2-b1f1-99404f330b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:53.123336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-2fb77808-cec9-42a2-b1f1-99404f330b1c None None] MFA Rules not processed for user `a9ac6a36e5224f38bdc5b214580ff18a`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:53.124615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-2fb77808-cec9-42a2-b1f1-99404f330b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:53.132434 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1396/2805] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jun 29 10:36:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:53.145593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8e431-e8f5-43ec-bbe3-dbce30229a9e service swift] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:53.158381 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8e431-e8f5-43ec-bbe3-dbce30229a9e service swift] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:53.164226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51c7fc-7f20-4323-b685-68d500d0483f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd86d073ef5ee4631b508ab578662be96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:53.165387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51c7fc-7f20-4323-b685-68d500d0483f service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:53.165387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51c7fc-7f20-4323-b685-68d500d0483f service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:53.165723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51c7fc-7f20-4323-b685-68d500d0483f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:53.176293 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51c7fc-7f20-4323-b685-68d500d0483f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:53.178049 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51c7fc-7f20-4323-b685-68d500d0483f service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:53.187911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1410/2806] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 10:36:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 10:36:55.361563 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb77808-cec9-42a2-b1f1-99404f330b1c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:55.438837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb77808-cec9-42a2-b1f1-99404f330b1c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:55.444815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:55.445946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:55.446530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:55.446923 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] PATH_INFO: `/v3/users/79681dafdc8546c2b641e10026360fee` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:55.448671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79681dafdc8546c2b641e10026360fee)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:55.451171 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:55.510461 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:55.512987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79681dafdc8546c2b641e10026360fee'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:55.536023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79681dafdc8546c2b641e10026360fee'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:55.537303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79681dafdc8546c2b641e10026360fee'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:55.538374 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79681dafdc8546c2b641e10026360fee'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:55.553890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79681dafdc8546c2b641e10026360fee'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:55.577593 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1397/2807] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:36:55 2022] DELETE /identity/v3/users/79681dafdc8546c2b641e10026360fee => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:36:55.583771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1b4d-111f-4635-b7aa-0b7aaa689d8a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:55.584563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1b4d-111f-4635-b7aa-0b7aaa689d8a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:55.584563 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1b4d-111f-4635-b7aa-0b7aaa689d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:55.604693 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1c9e1b4d-111f-4635-b7aa-0b7aaa689d8a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:36:55.605927 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1c9e1b4d-111f-4635-b7aa-0b7aaa689d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:55.617051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:36:55.678548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1411/2808] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:36:55.689149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:55.690852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:55.761023 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9caa3a-691b-4ffc-9108-3525db2b4fd6 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:55.765904 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f0adc-9fd2-466b-8b08-04db06a4483f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:55.766610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f0adc-9fd2-466b-8b08-04db06a4483f service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:55.766899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f0adc-9fd2-466b-8b08-04db06a4483f service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:55.767154 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f0adc-9fd2-466b-8b08-04db06a4483f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:55.778657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f0adc-9fd2-466b-8b08-04db06a4483f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:55.780131 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f0adc-9fd2-466b-8b08-04db06a4483f service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:55.791235 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1398/2809] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:36:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:36:55.992248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1b4d-111f-4635-b7aa-0b7aaa689d8a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:36:55.994313 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1c9e1b4d-111f-4635-b7aa-0b7aaa689d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:36:56.009619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1b4d-111f-4635-b7aa-0b7aaa689d8a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:36:56.019756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:36:56.021301 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:36:56.021989 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:36:56.022473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] PATH_INFO: `/v3/projects/556b0713c53744e1a58fed0756ac3a13` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:36:56.025349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=556b0713c53744e1a58fed0756ac3a13)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:36:56.029072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:36:56.086226 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '556b0713c53744e1a58fed0756ac3a13', 'request_id': 'req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:56.094370 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '556b0713c53744e1a58fed0756ac3a13', 'request_id': 'req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:56.101822 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '556b0713c53744e1a58fed0756ac3a13', 'request_id': 'req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:56.114743 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] The token cache is being invalidate because project 556b0713c53744e1a58fed0756ac3a13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:36:56.115029 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:56.115689 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:36:56.117244 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1412/2810] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:36:55 2022] DELETE /identity/v3/projects/556b0713c53744e1a58fed0756ac3a13 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:37:05.786136 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=39,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:37:06.111498 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:37:51.829070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f0adc-9fd2-466b-8b08-04db06a4483f service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:51.830756 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-652f0adc-9fd2-466b-8b08-04db06a4483f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:51.844317 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f0adc-9fd2-466b-8b08-04db06a4483f service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:51.901149 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc2c5d-db87-4655-ba74-21634eaafa00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:51.901781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc2c5d-db87-4655-ba74-21634eaafa00 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:51.902060 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc2c5d-db87-4655-ba74-21634eaafa00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:51.902333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc2c5d-db87-4655-ba74-21634eaafa00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:51.904381 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-46cc2c5d-db87-4655-ba74-21634eaafa00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:51.915965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc2c5d-db87-4655-ba74-21634eaafa00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:51.917471 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc2c5d-db87-4655-ba74-21634eaafa00 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:51.928110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1399/2811] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:37:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:37:52.045180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:52.046784 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:52.063999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94062e-377b-49e3-93c3-cbe02d9e1e8d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:52.072112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66730a-8875-4b75-9e40-26b9739fa1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:52.073212 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66730a-8875-4b75-9e40-26b9739fa1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:52.073708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66730a-8875-4b75-9e40-26b9739fa1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:52.074199 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66730a-8875-4b75-9e40-26b9739fa1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:52.077147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3b66730a-8875-4b75-9e40-26b9739fa1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:52.212074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66730a-8875-4b75-9e40-26b9739fa1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:52.213809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66730a-8875-4b75-9e40-26b9739fa1b0 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:52.229132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1413/2812] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:37:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:37:53.275126 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc2c5d-db87-4655-ba74-21634eaafa00 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:53.286530 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc2c5d-db87-4655-ba74-21634eaafa00 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:53.291958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4a74e-ee91-4a12-a990-a0292e2f7de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:53.293689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4a74e-ee91-4a12-a990-a0292e2f7de1 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:53.294255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4a74e-ee91-4a12-a990-a0292e2f7de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:53.294720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4a74e-ee91-4a12-a990-a0292e2f7de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:53.298875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-bdf4a74e-ee91-4a12-a990-a0292e2f7de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:53.384127 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4a74e-ee91-4a12-a990-a0292e2f7de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:53.385610 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4a74e-ee91-4a12-a990-a0292e2f7de1 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:53.400994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1400/2813] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 10:37:56.035999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66730a-8875-4b75-9e40-26b9739fa1b0 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:56.037545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3b66730a-8875-4b75-9e40-26b9739fa1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:56.155480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66730a-8875-4b75-9e40-26b9739fa1b0 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:56.162497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de49b09-eb95-4840-b330-4d11ccca352d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:56.172909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49b09-eb95-4840-b330-4d11ccca352d service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:56.172909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49b09-eb95-4840-b330-4d11ccca352d service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:56.172909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49b09-eb95-4840-b330-4d11ccca352d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:56.172909 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2de49b09-eb95-4840-b330-4d11ccca352d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:56.178594 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de49b09-eb95-4840-b330-4d11ccca352d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:56.181471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de49b09-eb95-4840-b330-4d11ccca352d service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:56.193539 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1414/2814] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:37:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:37:56.896330 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4a74e-ee91-4a12-a990-a0292e2f7de1 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:56.897917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-bdf4a74e-ee91-4a12-a990-a0292e2f7de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:56.911836 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4a74e-ee91-4a12-a990-a0292e2f7de1 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:56.919273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37536e8c-4f14-4f93-974b-6d3873c78dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:56.920256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37536e8c-4f14-4f93-974b-6d3873c78dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:56.920732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37536e8c-4f14-4f93-974b-6d3873c78dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:56.921196 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37536e8c-4f14-4f93-974b-6d3873c78dfd admin admin] PATH_INFO: `/v3/users/63ba0760d13e4bbda0650de803ef46ea` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:56.922737 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37536e8c-4f14-4f93-974b-6d3873c78dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ba0760d13e4bbda0650de803ef46ea)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:56.924779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37536e8c-4f14-4f93-974b-6d3873c78dfd admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:56.988726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-37536e8c-4f14-4f93-974b-6d3873c78dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ba0760d13e4bbda0650de803ef46ea'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:37:57.001785 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-37536e8c-4f14-4f93-974b-6d3873c78dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ba0760d13e4bbda0650de803ef46ea'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:37:57.002590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-37536e8c-4f14-4f93-974b-6d3873c78dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ba0760d13e4bbda0650de803ef46ea'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:37:57.003215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-37536e8c-4f14-4f93-974b-6d3873c78dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ba0760d13e4bbda0650de803ef46ea'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:37:57.011644 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-37536e8c-4f14-4f93-974b-6d3873c78dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ba0760d13e4bbda0650de803ef46ea'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:37:57.027130 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1401/2815] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:37:56 2022] DELETE /identity/v3/users/63ba0760d13e4bbda0650de803ef46ea => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:37:57.269863 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de49b09-eb95-4840-b330-4d11ccca352d service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:57.282351 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de49b09-eb95-4840-b330-4d11ccca352d service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:57.351757 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:57.352769 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:57.353050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:57.353328 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] PATH_INFO: `/v3/projects/c695a228acc84e93baf309bec5e489c4` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:57.354639 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c695a228acc84e93baf309bec5e489c4)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:57.356591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:57.375116 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:37:57.421855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c695a228acc84e93baf309bec5e489c4', 'request_id': 'req-89a05439-a4af-4ad3-901c-e5f82e6d40d7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:37:57.431210 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c695a228acc84e93baf309bec5e489c4', 'request_id': 'req-89a05439-a4af-4ad3-901c-e5f82e6d40d7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:37:57.437223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c695a228acc84e93baf309bec5e489c4', 'request_id': 'req-89a05439-a4af-4ad3-901c-e5f82e6d40d7'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:37:57.449847 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] The token cache is being invalidate because project c695a228acc84e93baf309bec5e489c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:37:57.450245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:37:57.451024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a05439-a4af-4ad3-901c-e5f82e6d40d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:37:57.452699 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1415/2816] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:37:57 2022] DELETE /identity/v3/projects/c695a228acc84e93baf309bec5e489c4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:37:57.458955 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350a448-37af-43e3-b2f1-5c58ebd206c9 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:57.459345 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350a448-37af-43e3-b2f1-5c58ebd206c9 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:57.459702 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350a448-37af-43e3-b2f1-5c58ebd206c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:57.478995 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6350a448-37af-43e3-b2f1-5c58ebd206c9 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:37:57.480298 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6350a448-37af-43e3-b2f1-5c58ebd206c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:57.484696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:37:57.560456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1402/2817] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:37:57.569245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd127b0f-77e3-42c2-8d78-9cff516fc5b7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:57.569541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd127b0f-77e3-42c2-8d78-9cff516fc5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:57.569835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd127b0f-77e3-42c2-8d78-9cff516fc5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:57.589590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-cd127b0f-77e3-42c2-8d78-9cff516fc5b7 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:37:57.590859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-cd127b0f-77e3-42c2-8d78-9cff516fc5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:57.599680 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1416/2818] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:37:57.604417 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350a448-37af-43e3-b2f1-5c58ebd206c9 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:57.615661 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350a448-37af-43e3-b2f1-5c58ebd206c9 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:57.621754 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bd28-a874-4155-a283-5146b2c7f4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:57.622440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bd28-a874-4155-a283-5146b2c7f4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:57.622755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bd28-a874-4155-a283-5146b2c7f4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:57.623030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bd28-a874-4155-a283-5146b2c7f4a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:57.623761 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455bd28-a874-4155-a283-5146b2c7f4a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:37:57.624093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455bd28-a874-4155-a283-5146b2c7f4a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:57.625719 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455bd28-a874-4155-a283-5146b2c7f4a3 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:57.638897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1403/2819] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:37:57.643514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd127b0f-77e3-42c2-8d78-9cff516fc5b7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:57.653634 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd127b0f-77e3-42c2-8d78-9cff516fc5b7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:57.659617 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91df68-9139-4390-a55c-0a8f6b061baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:57.660299 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91df68-9139-4390-a55c-0a8f6b061baa admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:57.660580 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91df68-9139-4390-a55c-0a8f6b061baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:57.660851 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91df68-9139-4390-a55c-0a8f6b061baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:57.661551 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91df68-9139-4390-a55c-0a8f6b061baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:57.663558 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91df68-9139-4390-a55c-0a8f6b061baa admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:57.693848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1417/2820] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:37:57 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:37:57.698477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bd28-a874-4155-a283-5146b2c7f4a3 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:57.709223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bd28-a874-4155-a283-5146b2c7f4a3 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:57.772712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19670089-9a0a-4654-8ac9-468c5e6f16f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:57.773389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19670089-9a0a-4654-8ac9-468c5e6f16f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:57.773671 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19670089-9a0a-4654-8ac9-468c5e6f16f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:57.773944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19670089-9a0a-4654-8ac9-468c5e6f16f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:57.774743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19670089-9a0a-4654-8ac9-468c5e6f16f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:57.776662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19670089-9a0a-4654-8ac9-468c5e6f16f8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:57.797981 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1404/2821] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:37:57 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:37:57.802335 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91df68-9139-4390-a55c-0a8f6b061baa admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:57.814120 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91df68-9139-4390-a55c-0a8f6b061baa admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:57.819497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f03dcf-b7e3-46f2-ae26-7b6883e88398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:57.820167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f03dcf-b7e3-46f2-ae26-7b6883e88398 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:57.820444 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f03dcf-b7e3-46f2-ae26-7b6883e88398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:57.820702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f03dcf-b7e3-46f2-ae26-7b6883e88398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:57.821541 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f03dcf-b7e3-46f2-ae26-7b6883e88398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:57.823267 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f03dcf-b7e3-46f2-ae26-7b6883e88398 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:57.834516 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1418/2822] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:37:57.839085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19670089-9a0a-4654-8ac9-468c5e6f16f8 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:57.849320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19670089-9a0a-4654-8ac9-468c5e6f16f8 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:57.854449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2b162-4d78-4a04-aadc-56b7791a006d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:57.855155 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2b162-4d78-4a04-aadc-56b7791a006d admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:57.855419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2b162-4d78-4a04-aadc-56b7791a006d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:57.855711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2b162-4d78-4a04-aadc-56b7791a006d admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:57.856440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2b162-4d78-4a04-aadc-56b7791a006d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:57.858006 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2b162-4d78-4a04-aadc-56b7791a006d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:57.868560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1405/2823] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:37:57.872826 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f03dcf-b7e3-46f2-ae26-7b6883e88398 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:57.883422 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f03dcf-b7e3-46f2-ae26-7b6883e88398 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:57.889074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bf553-720b-48b0-b89d-27a89172a408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:57.889963 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bf553-720b-48b0-b89d-27a89172a408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:57.890248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bf553-720b-48b0-b89d-27a89172a408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:57.890513 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bf553-720b-48b0-b89d-27a89172a408 admin admin] PATH_INFO: `/v3/projects/1328f3aff9a042068380feeb1e7c5f1e/users/066254622cf34ca282883c1a033722a4/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:57.906233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bf553-720b-48b0-b89d-27a89172a408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1328f3aff9a042068380feeb1e7c5f1e, user_id=066254622cf34ca282883c1a033722a4, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:57.909017 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bf553-720b-48b0-b89d-27a89172a408 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:57.924074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1419/2824] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:37:57 2022] PUT /identity/v3/projects/1328f3aff9a042068380feeb1e7c5f1e/users/066254622cf34ca282883c1a033722a4/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:37:57.929321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0747cd-84bc-4f91-93e3-9f55ed69cc22 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:57.930193 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0747cd-84bc-4f91-93e3-9f55ed69cc22 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:57.930385 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0747cd-84bc-4f91-93e3-9f55ed69cc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:57.949129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-6c0747cd-84bc-4f91-93e3-9f55ed69cc22 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:37:57.950332 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-6c0747cd-84bc-4f91-93e3-9f55ed69cc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:58.014867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1406/2825] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:37:58.025013 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bf553-720b-48b0-b89d-27a89172a408 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:37:58.027372 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9a7bf553-720b-48b0-b89d-27a89172a408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:37:58.092809 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bf553-720b-48b0-b89d-27a89172a408 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:37:58.096720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46f692-8d6a-4437-8078-c870b1b06139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:37:58.097559 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46f692-8d6a-4437-8078-c870b1b06139 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:37:58.097886 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46f692-8d6a-4437-8078-c870b1b06139 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:37:58.098160 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46f692-8d6a-4437-8078-c870b1b06139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:37:58.109164 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46f692-8d6a-4437-8078-c870b1b06139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:37:58.110653 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46f692-8d6a-4437-8078-c870b1b06139 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:37:58.121600 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1420/2826] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:38:03.692321 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54548ad3-d100-428e-aa61-c453b9dbd80b None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:38:03.692889 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54548ad3-d100-428e-aa61-c453b9dbd80b None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:38:03.693096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54548ad3-d100-428e-aa61-c453b9dbd80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:38:03.726950 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-54548ad3-d100-428e-aa61-c453b9dbd80b None None] MFA Rules not processed for user `066254622cf34ca282883c1a033722a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:38:03.728287 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-54548ad3-d100-428e-aa61-c453b9dbd80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:38:03.732788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:38:03.807980 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1407/2827] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:38:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:38:03.823375 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46f692-8d6a-4437-8078-c870b1b06139 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:38:03.825020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2e46f692-8d6a-4437-8078-c870b1b06139 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:38:03.880177 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:38:03.918444 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46f692-8d6a-4437-8078-c870b1b06139 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:38:03.932333 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d23671-27a2-44fc-a63a-061e61efe118 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:38:03.933020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d23671-27a2-44fc-a63a-061e61efe118 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:38:03.933303 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d23671-27a2-44fc-a63a-061e61efe118 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:38:03.933576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d23671-27a2-44fc-a63a-061e61efe118 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:38:03.947151 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d23671-27a2-44fc-a63a-061e61efe118 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:38:03.949488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d23671-27a2-44fc-a63a-061e61efe118 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:38:03.959440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1421/2828] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:38:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:38:05.378208 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17c524-6318-43f7-8121-00fc7d6b26e4 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:38:05.378786 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17c524-6318-43f7-8121-00fc7d6b26e4 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:38:05.378875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17c524-6318-43f7-8121-00fc7d6b26e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:38:05.402691 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fa17c524-6318-43f7-8121-00fc7d6b26e4 None None] MFA Rules not processed for user `066254622cf34ca282883c1a033722a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:38:05.403947 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fa17c524-6318-43f7-8121-00fc7d6b26e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:38:05.413928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1408/2829] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:38:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:38:13.955238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=14 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:38:15.395927 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=28 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:39:23.033981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d23671-27a2-44fc-a63a-061e61efe118 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:39:23.035981 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a5d23671-27a2-44fc-a63a-061e61efe118 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:39:23.154131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d23671-27a2-44fc-a63a-061e61efe118 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:39:23.160014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3efc7-6052-4e51-b3df-d5022c2684c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:39:23.160668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3efc7-6052-4e51-b3df-d5022c2684c7 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:39:23.160948 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3efc7-6052-4e51-b3df-d5022c2684c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:39:23.161221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3efc7-6052-4e51-b3df-d5022c2684c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:39:23.163170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ddb3efc7-6052-4e51-b3df-d5022c2684c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:39:23.179179 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3efc7-6052-4e51-b3df-d5022c2684c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:39:23.181009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3efc7-6052-4e51-b3df-d5022c2684c7 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:39:23.192586 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1422/2830] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:39:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:39:26.591143 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17c524-6318-43f7-8121-00fc7d6b26e4 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:39:26.592699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fa17c524-6318-43f7-8121-00fc7d6b26e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:39:26.607679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17c524-6318-43f7-8121-00fc7d6b26e4 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:39:26.612379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:39:26.613257 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:39:26.613560 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:39:26.613894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] PATH_INFO: `/v3/users/dd9cb5379e6148c9b283fb26fc578c75` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:39:26.615098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd9cb5379e6148c9b283fb26fc578c75)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:39:26.617044 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:39:26.671429 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd9cb5379e6148c9b283fb26fc578c75'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:26.685017 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd9cb5379e6148c9b283fb26fc578c75'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:26.685998 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd9cb5379e6148c9b283fb26fc578c75'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:26.688397 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd9cb5379e6148c9b283fb26fc578c75'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:26.697377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd9cb5379e6148c9b283fb26fc578c75'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:26.713728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1409/2831] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:39:26 2022] DELETE /identity/v3/users/dd9cb5379e6148c9b283fb26fc578c75 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:39:26.908700 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3efc7-6052-4e51-b3df-d5022c2684c7 service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:39:26.926901 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3efc7-6052-4e51-b3df-d5022c2684c7 service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:39:26.986208 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:39:26.987119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:39:26.987627 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:39:26.987899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] PATH_INFO: `/v3/projects/b73bddd1e4434ae6980546b225f2b341` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:39:26.989159 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73bddd1e4434ae6980546b225f2b341)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:39:26.991154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:39:27.044163 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73bddd1e4434ae6980546b225f2b341', 'request_id': 'req-e2024a44-59e8-4952-92b5-bd446283695a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.052576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73bddd1e4434ae6980546b225f2b341', 'request_id': 'req-e2024a44-59e8-4952-92b5-bd446283695a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.059507 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73bddd1e4434ae6980546b225f2b341', 'request_id': 'req-e2024a44-59e8-4952-92b5-bd446283695a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.070967 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] The token cache is being invalidate because project b73bddd1e4434ae6980546b225f2b341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:39:27.071249 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.072068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.073667 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1423/2832] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:39:26 2022] DELETE /identity/v3/projects/b73bddd1e4434ae6980546b225f2b341 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:39:27.078521 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:39:27.081227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:39:27.104949 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444d1f0-b64a-4c8e-a8fc-33bebbb53a46 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:39:27.172669 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:39:27.173568 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:39:27.173868 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:39:27.174139 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] PATH_INFO: `/v3/users/edab531bbacd47fd92cf49bef7b8a740` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:39:27.175467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edab531bbacd47fd92cf49bef7b8a740)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:39:27.177467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:39:27.236129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edab531bbacd47fd92cf49bef7b8a740'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.252506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edab531bbacd47fd92cf49bef7b8a740'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.253698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edab531bbacd47fd92cf49bef7b8a740'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.254654 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edab531bbacd47fd92cf49bef7b8a740'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.265535 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edab531bbacd47fd92cf49bef7b8a740'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.268034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:39:27.285102 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1410/2833] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:39:27 2022] DELETE /identity/v3/users/edab531bbacd47fd92cf49bef7b8a740 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:39:27.478060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:39:27.479641 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:39:27.496592 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2024a44-59e8-4952-92b5-bd446283695a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:39:27.521352 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:39:27.566176 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:39:27.567101 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:39:27.567384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:39:27.567673 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] PATH_INFO: `/v3/projects/2e8464a3ee6646f88c02e93fca5a69bd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:39:27.569321 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e8464a3ee6646f88c02e93fca5a69bd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:39:27.572131 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:39:27.636051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e8464a3ee6646f88c02e93fca5a69bd', 'request_id': 'req-5e43d633-9ec1-464e-a860-151f815b8bef'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.645263 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e8464a3ee6646f88c02e93fca5a69bd', 'request_id': 'req-5e43d633-9ec1-464e-a860-151f815b8bef'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.651675 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e8464a3ee6646f88c02e93fca5a69bd', 'request_id': 'req-5e43d633-9ec1-464e-a860-151f815b8bef'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.667213 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] The token cache is being invalidate because project 2e8464a3ee6646f88c02e93fca5a69bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:39:27.667493 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.668157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:39:27.669723 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1424/2834] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:39:27 2022] DELETE /identity/v3/projects/2e8464a3ee6646f88c02e93fca5a69bd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:39:37.282342 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:39:37.664187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:40:04.437187 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:40:04.438763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:04.506687 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba5caa-be04-43f6-a116-93febb5d3a89 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:40:04.558618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46181b-629c-4443-94d7-cf95f8e08d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:40:04.559291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46181b-629c-4443-94d7-cf95f8e08d19 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:40:04.559575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46181b-629c-4443-94d7-cf95f8e08d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:40:04.559852 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46181b-629c-4443-94d7-cf95f8e08d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:40:04.561861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4d46181b-629c-4443-94d7-cf95f8e08d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:04.572370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46181b-629c-4443-94d7-cf95f8e08d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:40:04.573837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46181b-629c-4443-94d7-cf95f8e08d19 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:40:04.582520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1411/2835] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:40:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:40:14.578575 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=46 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:40:38.200420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:40:38.202209 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:38.216442 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43d633-9ec1-464e-a860-151f815b8bef admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:40:38.220375 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ffd78-6845-45ec-af07-7cc7a96c2997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:40:38.221034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ffd78-6845-45ec-af07-7cc7a96c2997 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:40:38.221313 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ffd78-6845-45ec-af07-7cc7a96c2997 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:40:38.221583 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ffd78-6845-45ec-af07-7cc7a96c2997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:40:38.223490 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-130ffd78-6845-45ec-af07-7cc7a96c2997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:38.298146 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ffd78-6845-45ec-af07-7cc7a96c2997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:40:38.299628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ffd78-6845-45ec-af07-7cc7a96c2997 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:40:38.312468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1425/2836] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:40:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:40:39.505303 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46181b-629c-4443-94d7-cf95f8e08d19 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:40:39.507728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4d46181b-629c-4443-94d7-cf95f8e08d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:39.518689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46181b-629c-4443-94d7-cf95f8e08d19 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:40:39.522626 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b86764-f6b2-40ee-8d9a-386c906ebf4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:40:39.523295 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b86764-f6b2-40ee-8d9a-386c906ebf4c service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:40:39.523574 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b86764-f6b2-40ee-8d9a-386c906ebf4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:40:39.523846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b86764-f6b2-40ee-8d9a-386c906ebf4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:40:39.525680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-48b86764-f6b2-40ee-8d9a-386c906ebf4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:39.546106 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b86764-f6b2-40ee-8d9a-386c906ebf4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:40:39.548794 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b86764-f6b2-40ee-8d9a-386c906ebf4c service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:40:39.564105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1412/2837] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:40:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:40:43.874278 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46af3ea-c4ba-45e9-bfa3-99a5d92610b3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:40:43.874929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46af3ea-c4ba-45e9-bfa3-99a5d92610b3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:40:43.874929 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46af3ea-c4ba-45e9-bfa3-99a5d92610b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:40:43.897939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b46af3ea-c4ba-45e9-bfa3-99a5d92610b3 None None] MFA Rules not processed for user `066254622cf34ca282883c1a033722a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:40:43.899377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b46af3ea-c4ba-45e9-bfa3-99a5d92610b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:43.964579 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1426/2838] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:40:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:40:44.302190 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b86764-f6b2-40ee-8d9a-386c906ebf4c service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:40:44.312462 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b86764-f6b2-40ee-8d9a-386c906ebf4c service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:40:44.316759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f4cbe-7cff-4a5d-b648-4f9218317aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:40:44.317406 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f4cbe-7cff-4a5d-b648-4f9218317aaf service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:40:44.317680 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f4cbe-7cff-4a5d-b648-4f9218317aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:40:44.317948 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f4cbe-7cff-4a5d-b648-4f9218317aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:40:44.320056 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ea4f4cbe-7cff-4a5d-b648-4f9218317aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:44.330897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f4cbe-7cff-4a5d-b648-4f9218317aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:40:44.332323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f4cbe-7cff-4a5d-b648-4f9218317aaf service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:40:44.341226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1413/2839] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:40:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:40:44.680841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46af3ea-c4ba-45e9-bfa3-99a5d92610b3 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:40:44.682593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b46af3ea-c4ba-45e9-bfa3-99a5d92610b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:44.775115 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46af3ea-c4ba-45e9-bfa3-99a5d92610b3 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:40:44.782405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40901d70-da7f-4487-9493-095f5f42065b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:40:44.783098 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40901d70-da7f-4487-9493-095f5f42065b service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:40:44.783376 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40901d70-da7f-4487-9493-095f5f42065b service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:40:44.783647 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40901d70-da7f-4487-9493-095f5f42065b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:40:44.785696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-40901d70-da7f-4487-9493-095f5f42065b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:44.799010 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40901d70-da7f-4487-9493-095f5f42065b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:40:44.800548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40901d70-da7f-4487-9493-095f5f42065b service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:40:44.811542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1427/2840] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:40:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:40:52.299157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f4cbe-7cff-4a5d-b648-4f9218317aaf service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:40:52.300755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ea4f4cbe-7cff-4a5d-b648-4f9218317aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:52.319419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f4cbe-7cff-4a5d-b648-4f9218317aaf service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:40:52.327292 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcf6f1-9754-4da3-bf72-9da9f4722405 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:40:52.327966 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf6f1-9754-4da3-bf72-9da9f4722405 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:40:52.328246 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf6f1-9754-4da3-bf72-9da9f4722405 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:40:52.328518 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf6f1-9754-4da3-bf72-9da9f4722405 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:40:52.330307 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7cdcf6f1-9754-4da3-bf72-9da9f4722405 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:52.343258 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcf6f1-9754-4da3-bf72-9da9f4722405 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:40:52.344853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcf6f1-9754-4da3-bf72-9da9f4722405 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:40:52.356219 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1414/2841] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:40:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:40:54.806976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=91 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:40:55.573405 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40901d70-da7f-4487-9493-095f5f42065b service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:40:55.586771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40901d70-da7f-4487-9493-095f5f42065b service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:40:55.593011 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42829071-6a12-4efb-8bbd-a1080386d79c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:40:55.593690 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42829071-6a12-4efb-8bbd-a1080386d79c service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:40:55.593973 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42829071-6a12-4efb-8bbd-a1080386d79c service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:40:55.594305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42829071-6a12-4efb-8bbd-a1080386d79c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:40:55.596250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-42829071-6a12-4efb-8bbd-a1080386d79c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:40:55.609645 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42829071-6a12-4efb-8bbd-a1080386d79c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:40:55.611229 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42829071-6a12-4efb-8bbd-a1080386d79c service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:40:55.622803 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1428/2842] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:40:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:41:02.351657 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=18 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:41:05.617745 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:41:54.382016 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcf6f1-9754-4da3-bf72-9da9f4722405 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:41:54.383547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7cdcf6f1-9754-4da3-bf72-9da9f4722405 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:41:54.398738 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcf6f1-9754-4da3-bf72-9da9f4722405 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:41:54.405322 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e523eb-8949-4f30-8eef-9a173c292741 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5eea7f539048c48ed10e72dd933df9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:41:54.405931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e523eb-8949-4f30-8eef-9a173c292741 service nova] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:41:54.406206 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e523eb-8949-4f30-8eef-9a173c292741 service nova] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:41:54.406493 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e523eb-8949-4f30-8eef-9a173c292741 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:41:54.408463 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a0e523eb-8949-4f30-8eef-9a173c292741 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:41:54.422450 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e523eb-8949-4f30-8eef-9a173c292741 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:41:54.424113 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e523eb-8949-4f30-8eef-9a173c292741 service nova] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:41:54.438173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1415/2843] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:41:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:02.789266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42829071-6a12-4efb-8bbd-a1080386d79c service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:02.791153 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-42829071-6a12-4efb-8bbd-a1080386d79c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:02.866184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42829071-6a12-4efb-8bbd-a1080386d79c service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:02.871413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785bfd8-0f63-4e7a-92fb-52485ceee1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:02.872307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785bfd8-0f63-4e7a-92fb-52485ceee1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:02.872583 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785bfd8-0f63-4e7a-92fb-52485ceee1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:02.872849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785bfd8-0f63-4e7a-92fb-52485ceee1af admin admin] PATH_INFO: `/v3/users/066254622cf34ca282883c1a033722a4` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:02.874170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785bfd8-0f63-4e7a-92fb-52485ceee1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=066254622cf34ca282883c1a033722a4)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:02.876147 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785bfd8-0f63-4e7a-92fb-52485ceee1af admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:02.944890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9785bfd8-0f63-4e7a-92fb-52485ceee1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '066254622cf34ca282883c1a033722a4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:02.962519 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9785bfd8-0f63-4e7a-92fb-52485ceee1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '066254622cf34ca282883c1a033722a4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:02.963323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9785bfd8-0f63-4e7a-92fb-52485ceee1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '066254622cf34ca282883c1a033722a4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:02.970821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9785bfd8-0f63-4e7a-92fb-52485ceee1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '066254622cf34ca282883c1a033722a4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:02.971539 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9785bfd8-0f63-4e7a-92fb-52485ceee1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '066254622cf34ca282883c1a033722a4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:02.985621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1429/2844] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:42:02 2022] DELETE /identity/v3/users/066254622cf34ca282883c1a033722a4 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:03.192252 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e523eb-8949-4f30-8eef-9a173c292741 service nova] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:03.202977 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e523eb-8949-4f30-8eef-9a173c292741 service nova] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:03.276890 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:03.279035 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:03.279525 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:03.279931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] PATH_INFO: `/v3/projects/1328f3aff9a042068380feeb1e7c5f1e` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:03.282002 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1328f3aff9a042068380feeb1e7c5f1e)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:03.285047 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:03.346555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1328f3aff9a042068380feeb1e7c5f1e', 'request_id': 'req-3283b68b-afba-48da-8376-8e90b3d2f5b8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:03.356115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1328f3aff9a042068380feeb1e7c5f1e', 'request_id': 'req-3283b68b-afba-48da-8376-8e90b3d2f5b8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:03.362862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1328f3aff9a042068380feeb1e7c5f1e', 'request_id': 'req-3283b68b-afba-48da-8376-8e90b3d2f5b8'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:03.374440 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] The token cache is being invalidate because project 1328f3aff9a042068380feeb1e7c5f1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:42:03.374748 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:03.375776 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-3283b68b-afba-48da-8376-8e90b3d2f5b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:03.377369 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1416/2845] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:42:03 2022] DELETE /identity/v3/projects/1328f3aff9a042068380feeb1e7c5f1e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:03.382489 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a3547-703b-45cf-98e7-f4e2adb6d658 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:03.382830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a3547-703b-45cf-98e7-f4e2adb6d658 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:03.383083 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a3547-703b-45cf-98e7-f4e2adb6d658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:03.403021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c41a3547-703b-45cf-98e7-f4e2adb6d658 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:03.404478 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c41a3547-703b-45cf-98e7-f4e2adb6d658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:03.480203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1430/2846] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:03.488506 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5070fc-838c-4ffa-b0de-f47ea311233e None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:03.488788 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5070fc-838c-4ffa-b0de-f47ea311233e None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:03.489063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5070fc-838c-4ffa-b0de-f47ea311233e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:03.508838 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-9d5070fc-838c-4ffa-b0de-f47ea311233e None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:03.510042 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9d5070fc-838c-4ffa-b0de-f47ea311233e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:03.520939 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1417/2847] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:03.525412 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a3547-703b-45cf-98e7-f4e2adb6d658 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:03.536366 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a3547-703b-45cf-98e7-f4e2adb6d658 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:03.542520 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ef960-a325-4e2b-be42-8be823d54270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:03.543228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ef960-a325-4e2b-be42-8be823d54270 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:03.543510 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ef960-a325-4e2b-be42-8be823d54270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:03.543780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ef960-a325-4e2b-be42-8be823d54270 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:03.544497 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ef960-a325-4e2b-be42-8be823d54270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:42:03.544800 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ef960-a325-4e2b-be42-8be823d54270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:03.546420 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ef960-a325-4e2b-be42-8be823d54270 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:03.560079 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1431/2848] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:42:03.564877 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5070fc-838c-4ffa-b0de-f47ea311233e None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:03.574771 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5070fc-838c-4ffa-b0de-f47ea311233e None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:03.579953 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b4b49-9ded-4057-8c1d-d7b199faa400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:03.580627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b4b49-9ded-4057-8c1d-d7b199faa400 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:03.580905 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b4b49-9ded-4057-8c1d-d7b199faa400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:03.581185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b4b49-9ded-4057-8c1d-d7b199faa400 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:03.581883 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b4b49-9ded-4057-8c1d-d7b199faa400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:03.583821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b4b49-9ded-4057-8c1d-d7b199faa400 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:03.624711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1418/2849] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:42:03 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:03.629473 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ef960-a325-4e2b-be42-8be823d54270 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:03.640214 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ef960-a325-4e2b-be42-8be823d54270 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:03.644503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:03.709780 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91fbb4-7138-48ac-bb6e-008e0c2e49ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:03.710539 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91fbb4-7138-48ac-bb6e-008e0c2e49ee admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:03.710998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91fbb4-7138-48ac-bb6e-008e0c2e49ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:03.711269 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91fbb4-7138-48ac-bb6e-008e0c2e49ee admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:03.711968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91fbb4-7138-48ac-bb6e-008e0c2e49ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:03.713926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91fbb4-7138-48ac-bb6e-008e0c2e49ee admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:03.735450 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1432/2850] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:42:03 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:03.739682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b4b49-9ded-4057-8c1d-d7b199faa400 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:03.751793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b4b49-9ded-4057-8c1d-d7b199faa400 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:03.757844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f645a2b-4b42-4043-9622-351c19e1761c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:03.758529 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f645a2b-4b42-4043-9622-351c19e1761c admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:03.758830 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f645a2b-4b42-4043-9622-351c19e1761c admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:03.759110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f645a2b-4b42-4043-9622-351c19e1761c admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:03.759861 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f645a2b-4b42-4043-9622-351c19e1761c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:03.761465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f645a2b-4b42-4043-9622-351c19e1761c admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:03.773418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1419/2851] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:03.777628 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91fbb4-7138-48ac-bb6e-008e0c2e49ee admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:03.788470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91fbb4-7138-48ac-bb6e-008e0c2e49ee admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:03.793597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95304044-641c-4115-8884-9e5252e4dee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:03.794281 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95304044-641c-4115-8884-9e5252e4dee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:03.794580 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95304044-641c-4115-8884-9e5252e4dee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:03.794865 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95304044-641c-4115-8884-9e5252e4dee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:03.795602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95304044-641c-4115-8884-9e5252e4dee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:03.797262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95304044-641c-4115-8884-9e5252e4dee3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:03.809064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1433/2852] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:03.813743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f645a2b-4b42-4043-9622-351c19e1761c admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:03.826098 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f645a2b-4b42-4043-9622-351c19e1761c admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:03.831465 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc95ec3-e3aa-401f-bfe0-4f020170f0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:03.832372 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc95ec3-e3aa-401f-bfe0-4f020170f0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:03.832811 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc95ec3-e3aa-401f-bfe0-4f020170f0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:03.833096 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc95ec3-e3aa-401f-bfe0-4f020170f0f4 admin admin] PATH_INFO: `/v3/projects/2790e19b362a410b9fe8a5db8d5db4a6/users/3e55165be86c41bc9b9c930e2241ca5e/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:03.850472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc95ec3-e3aa-401f-bfe0-4f020170f0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2790e19b362a410b9fe8a5db8d5db4a6, user_id=3e55165be86c41bc9b9c930e2241ca5e, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:03.853195 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc95ec3-e3aa-401f-bfe0-4f020170f0f4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:03.871503 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1420/2853] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:03 2022] PUT /identity/v3/projects/2790e19b362a410b9fe8a5db8d5db4a6/users/3e55165be86c41bc9b9c930e2241ca5e/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:03.935373 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62ead3-eb89-4f55-b7e1-070cf3174c89 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:03.935806 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62ead3-eb89-4f55-b7e1-070cf3174c89 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:03.936184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62ead3-eb89-4f55-b7e1-070cf3174c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:03.969581 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6d62ead3-eb89-4f55-b7e1-070cf3174c89 None None] MFA Rules not processed for user `3e55165be86c41bc9b9c930e2241ca5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:03.970841 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6d62ead3-eb89-4f55-b7e1-070cf3174c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:04.045362 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1434/2854] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:04.050867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc95ec3-e3aa-401f-bfe0-4f020170f0f4 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:04.073157 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:04.129731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc95ec3-e3aa-401f-bfe0-4f020170f0f4 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:04.135038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27398b-909c-408a-a12c-57e9019abada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:04.135724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27398b-909c-408a-a12c-57e9019abada admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:04.135960 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27398b-909c-408a-a12c-57e9019abada admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:04.136242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27398b-909c-408a-a12c-57e9019abada admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:04.136946 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27398b-909c-408a-a12c-57e9019abada admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:04.138978 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27398b-909c-408a-a12c-57e9019abada admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:04.170590 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1421/2855] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:42:04 2022] POST /identity/v3/projects => generated 389 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:04.175250 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62ead3-eb89-4f55-b7e1-070cf3174c89 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:04.186695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62ead3-eb89-4f55-b7e1-070cf3174c89 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:04.250528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28e2b0-b0a5-4a21-bd9a-450c08eb47a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:04.251044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28e2b0-b0a5-4a21-bd9a-450c08eb47a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:04.251326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28e2b0-b0a5-4a21-bd9a-450c08eb47a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:04.251604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28e2b0-b0a5-4a21-bd9a-450c08eb47a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:04.252307 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28e2b0-b0a5-4a21-bd9a-450c08eb47a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:04.254259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28e2b0-b0a5-4a21-bd9a-450c08eb47a8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:04.277602 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1435/2856] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:42:04 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:04.282513 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27398b-909c-408a-a12c-57e9019abada admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:04.293952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27398b-909c-408a-a12c-57e9019abada admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:04.299599 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8990cc5-b736-4125-ae40-54666ce9aa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:04.300225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8990cc5-b736-4125-ae40-54666ce9aa16 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:04.300505 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8990cc5-b736-4125-ae40-54666ce9aa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:04.300775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8990cc5-b736-4125-ae40-54666ce9aa16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:04.301851 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8990cc5-b736-4125-ae40-54666ce9aa16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:04.303490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8990cc5-b736-4125-ae40-54666ce9aa16 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:04.315333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1422/2857] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:04.319761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28e2b0-b0a5-4a21-bd9a-450c08eb47a8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:04.326474 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:04.330890 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28e2b0-b0a5-4a21-bd9a-450c08eb47a8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:04.337047 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0382d9b-dbb4-49dd-8be8-b6ea87902e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:04.337980 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0382d9b-dbb4-49dd-8be8-b6ea87902e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:04.338245 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0382d9b-dbb4-49dd-8be8-b6ea87902e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:04.338518 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0382d9b-dbb4-49dd-8be8-b6ea87902e51 admin admin] PATH_INFO: `/v3/projects/803271f192854753abffc9825e49220e/users/a6252760e34b42faabd12e127d20cea5/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:04.355391 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0382d9b-dbb4-49dd-8be8-b6ea87902e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=803271f192854753abffc9825e49220e, user_id=a6252760e34b42faabd12e127d20cea5, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:04.358077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0382d9b-dbb4-49dd-8be8-b6ea87902e51 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:04.373831 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1436/2858] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:04 2022] PUT /identity/v3/projects/803271f192854753abffc9825e49220e/users/a6252760e34b42faabd12e127d20cea5/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:04.378181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8990cc5-b736-4125-ae40-54666ce9aa16 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:04.388604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8990cc5-b736-4125-ae40-54666ce9aa16 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:04.451563 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ec0a1-d875-4bc0-aedf-c3de12bf3951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:04.452242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ec0a1-d875-4bc0-aedf-c3de12bf3951 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:04.452524 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ec0a1-d875-4bc0-aedf-c3de12bf3951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:04.452793 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ec0a1-d875-4bc0-aedf-c3de12bf3951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:04.453648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ec0a1-d875-4bc0-aedf-c3de12bf3951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:04.455352 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ec0a1-d875-4bc0-aedf-c3de12bf3951 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:04.465928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1423/2859] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:04.470496 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0382d9b-dbb4-49dd-8be8-b6ea87902e51 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:04.481708 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0382d9b-dbb4-49dd-8be8-b6ea87902e51 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:04.487817 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23332769-ba29-41d1-ac2a-d89ea00b02dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:04.488727 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332769-ba29-41d1-ac2a-d89ea00b02dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:04.489000 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332769-ba29-41d1-ac2a-d89ea00b02dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:04.489292 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332769-ba29-41d1-ac2a-d89ea00b02dc admin admin] PATH_INFO: `/v3/projects/803271f192854753abffc9825e49220e/users/a6252760e34b42faabd12e127d20cea5/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:04.491597 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23332769-ba29-41d1-ac2a-d89ea00b02dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=803271f192854753abffc9825e49220e, user_id=a6252760e34b42faabd12e127d20cea5, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:04.494305 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23332769-ba29-41d1-ac2a-d89ea00b02dc admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:04.506102 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1437/2860] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:04 2022] PUT /identity/v3/projects/803271f192854753abffc9825e49220e/users/a6252760e34b42faabd12e127d20cea5/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:04.516994 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4b736-ec45-4862-a26c-11b052dbaf19 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:04.517324 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4b736-ec45-4862-a26c-11b052dbaf19 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:04.517629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4b736-ec45-4862-a26c-11b052dbaf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:04.547012 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-dca4b736-ec45-4862-a26c-11b052dbaf19 None None] MFA Rules not processed for user `a6252760e34b42faabd12e127d20cea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:04.548917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-dca4b736-ec45-4862-a26c-11b052dbaf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:04.635197 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1424/2861] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:42:04.646768 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23332769-ba29-41d1-ac2a-d89ea00b02dc admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:04.648633 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-23332769-ba29-41d1-ac2a-d89ea00b02dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:04.714149 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23332769-ba29-41d1-ac2a-d89ea00b02dc admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:04.718088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da24be-5049-4c88-9c69-a5d8b8dc51ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:04.718808 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da24be-5049-4c88-9c69-a5d8b8dc51ed service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:04.719192 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da24be-5049-4c88-9c69-a5d8b8dc51ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:04.719503 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da24be-5049-4c88-9c69-a5d8b8dc51ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:04.729774 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da24be-5049-4c88-9c69-a5d8b8dc51ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:04.731273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da24be-5049-4c88-9c69-a5d8b8dc51ed service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:04.741734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1438/2862] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:04.881815 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4b736-ec45-4862-a26c-11b052dbaf19 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:04.895477 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:04.955965 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4b736-ec45-4862-a26c-11b052dbaf19 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:04.961934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:04.962910 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:04.963181 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:04.963446 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] PATH_INFO: `/v3/users/3e55165be86c41bc9b9c930e2241ca5e` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:04.964676 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e55165be86c41bc9b9c930e2241ca5e)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:04.966646 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:05.021651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e55165be86c41bc9b9c930e2241ca5e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:05.034922 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e55165be86c41bc9b9c930e2241ca5e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:05.035707 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e55165be86c41bc9b9c930e2241ca5e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:05.036385 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e55165be86c41bc9b9c930e2241ca5e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:05.044262 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e55165be86c41bc9b9c930e2241ca5e'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:05.060085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1425/2863] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:42:04 2022] DELETE /identity/v3/users/3e55165be86c41bc9b9c930e2241ca5e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:05.064710 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837df0d-7a9d-4c95-8944-47a32d355eae None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:05.065227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837df0d-7a9d-4c95-8944-47a32d355eae None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:05.065227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837df0d-7a9d-4c95-8944-47a32d355eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:05.085088 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9837df0d-7a9d-4c95-8944-47a32d355eae None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:05.087761 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9837df0d-7a9d-4c95-8944-47a32d355eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:05.148733 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1439/2864] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:05.158743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:05.160240 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:05.229988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf3a86-cf85-4d96-8b1c-e9eac2832a09 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:05.234359 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff7423-41b1-4fe6-9def-1c5016280c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:05.235115 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff7423-41b1-4fe6-9def-1c5016280c6d service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:05.235399 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff7423-41b1-4fe6-9def-1c5016280c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:05.235678 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff7423-41b1-4fe6-9def-1c5016280c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:05.252393 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff7423-41b1-4fe6-9def-1c5016280c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:05.253837 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff7423-41b1-4fe6-9def-1c5016280c6d service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:05.264435 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1426/2865] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:05.865946 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837df0d-7a9d-4c95-8944-47a32d355eae None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:05.867471 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9837df0d-7a9d-4c95-8944-47a32d355eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:05.880074 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837df0d-7a9d-4c95-8944-47a32d355eae None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:05.885463 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:05.885635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:05.885911 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:05.886273 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] PATH_INFO: `/v3/projects/2790e19b362a410b9fe8a5db8d5db4a6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:05.887557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2790e19b362a410b9fe8a5db8d5db4a6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:05.889488 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:05.943576 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2790e19b362a410b9fe8a5db8d5db4a6', 'request_id': 'req-2de84a01-33dd-4905-b614-2492d31e591a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:05.952713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2790e19b362a410b9fe8a5db8d5db4a6', 'request_id': 'req-2de84a01-33dd-4905-b614-2492d31e591a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:05.959022 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2790e19b362a410b9fe8a5db8d5db4a6', 'request_id': 'req-2de84a01-33dd-4905-b614-2492d31e591a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:05.960109 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:05.973677 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] The token cache is being invalidate because project 2790e19b362a410b9fe8a5db8d5db4a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:42:05.973957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:05.974682 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:05.976706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1440/2866] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:42:05 2022] DELETE /identity/v3/projects/2790e19b362a410b9fe8a5db8d5db4a6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:05.980853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff7423-41b1-4fe6-9def-1c5016280c6d service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:05.991679 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff7423-41b1-4fe6-9def-1c5016280c6d service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:06.024270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:06.060647 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018a7f7-9b7d-43c1-affb-05e7ee91d4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:06.061565 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018a7f7-9b7d-43c1-affb-05e7ee91d4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:06.061844 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018a7f7-9b7d-43c1-affb-05e7ee91d4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:06.062129 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018a7f7-9b7d-43c1-affb-05e7ee91d4ca admin admin] PATH_INFO: `/v3/users/a6252760e34b42faabd12e127d20cea5` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:06.064128 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018a7f7-9b7d-43c1-affb-05e7ee91d4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6252760e34b42faabd12e127d20cea5)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:06.066099 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018a7f7-9b7d-43c1-affb-05e7ee91d4ca admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:06.129986 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e018a7f7-9b7d-43c1-affb-05e7ee91d4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6252760e34b42faabd12e127d20cea5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:06.146520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e018a7f7-9b7d-43c1-affb-05e7ee91d4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6252760e34b42faabd12e127d20cea5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:06.147377 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e018a7f7-9b7d-43c1-affb-05e7ee91d4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6252760e34b42faabd12e127d20cea5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:06.148087 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e018a7f7-9b7d-43c1-affb-05e7ee91d4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6252760e34b42faabd12e127d20cea5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:06.156395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e018a7f7-9b7d-43c1-affb-05e7ee91d4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6252760e34b42faabd12e127d20cea5'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:06.170165 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1427/2867] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:42:05 2022] DELETE /identity/v3/users/a6252760e34b42faabd12e127d20cea5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:06.479737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:06.481308 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:06.494931 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de84a01-33dd-4905-b614-2492d31e591a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:06.563388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:06.564155 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:06.564459 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:06.564762 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] PATH_INFO: `/v3/projects/803271f192854753abffc9825e49220e` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:06.566072 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=803271f192854753abffc9825e49220e)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:06.568054 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:06.637401 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '803271f192854753abffc9825e49220e', 'request_id': 'req-32454136-1690-4f42-bece-84366c2a8b32'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:06.647056 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '803271f192854753abffc9825e49220e', 'request_id': 'req-32454136-1690-4f42-bece-84366c2a8b32'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:06.653815 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '803271f192854753abffc9825e49220e', 'request_id': 'req-32454136-1690-4f42-bece-84366c2a8b32'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:06.667079 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] The token cache is being invalidate because project 803271f192854753abffc9825e49220e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:42:06.667346 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:06.668033 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-32454136-1690-4f42-bece-84366c2a8b32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:06.669883 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1441/2868] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:42:06 2022] DELETE /identity/v3/projects/803271f192854753abffc9825e49220e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:06.675565 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436d037-17db-4bd7-a75a-c79cfaad0b52 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:06.675885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436d037-17db-4bd7-a75a-c79cfaad0b52 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:06.676132 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436d037-17db-4bd7-a75a-c79cfaad0b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:06.696243 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-3436d037-17db-4bd7-a75a-c79cfaad0b52 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:06.697490 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-3436d037-17db-4bd7-a75a-c79cfaad0b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:06.771929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1428/2869] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:06.779646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410abf5-99f6-467c-94b2-371d49b9dc85 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:06.779939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410abf5-99f6-467c-94b2-371d49b9dc85 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:06.780228 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410abf5-99f6-467c-94b2-371d49b9dc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:06.801181 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a410abf5-99f6-467c-94b2-371d49b9dc85 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:06.802789 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a410abf5-99f6-467c-94b2-371d49b9dc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:06.811021 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1442/2870] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:06.815746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436d037-17db-4bd7-a75a-c79cfaad0b52 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:06.826984 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436d037-17db-4bd7-a75a-c79cfaad0b52 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:06.834876 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315a872-4c1c-41aa-8453-320ef53adc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:06.835721 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315a872-4c1c-41aa-8453-320ef53adc5e admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:06.836255 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315a872-4c1c-41aa-8453-320ef53adc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:06.836608 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315a872-4c1c-41aa-8453-320ef53adc5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:06.837449 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315a872-4c1c-41aa-8453-320ef53adc5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:42:06.837855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315a872-4c1c-41aa-8453-320ef53adc5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:06.839627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315a872-4c1c-41aa-8453-320ef53adc5e admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:06.858360 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1429/2871] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:42:06.859630 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410abf5-99f6-467c-94b2-371d49b9dc85 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:06.871663 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410abf5-99f6-467c-94b2-371d49b9dc85 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:06.876859 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e8bf8-ad0e-4e53-95ec-df5b3b4e9ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:06.877610 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e8bf8-ad0e-4e53-95ec-df5b3b4e9ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:06.877903 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e8bf8-ad0e-4e53-95ec-df5b3b4e9ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:06.878204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e8bf8-ad0e-4e53-95ec-df5b3b4e9ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:06.878920 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e8bf8-ad0e-4e53-95ec-df5b3b4e9ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:06.880846 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e8bf8-ad0e-4e53-95ec-df5b3b4e9ba3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:06.917449 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1443/2872] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:42:06 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:42:06.921816 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315a872-4c1c-41aa-8453-320ef53adc5e admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:06.935547 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315a872-4c1c-41aa-8453-320ef53adc5e admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:06.972291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:06.998587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66702dc0-caf4-421b-a739-a84fc7c52a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:06.999320 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66702dc0-caf4-421b-a739-a84fc7c52a21 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:06.999546 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66702dc0-caf4-421b-a739-a84fc7c52a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:06.999814 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66702dc0-caf4-421b-a739-a84fc7c52a21 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.000614 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66702dc0-caf4-421b-a739-a84fc7c52a21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:07.002536 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66702dc0-caf4-421b-a739-a84fc7c52a21 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:07.023520 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1430/2873] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:42:06 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:07.027755 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e8bf8-ad0e-4e53-95ec-df5b3b4e9ba3 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:07.038409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e8bf8-ad0e-4e53-95ec-df5b3b4e9ba3 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:07.043092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689ffac-6ac7-4920-b531-374333332cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:07.043771 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689ffac-6ac7-4920-b531-374333332cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.044052 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689ffac-6ac7-4920-b531-374333332cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.044353 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689ffac-6ac7-4920-b531-374333332cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.045060 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689ffac-6ac7-4920-b531-374333332cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:07.046706 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689ffac-6ac7-4920-b531-374333332cc5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:07.058843 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1444/2874] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:07.062971 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66702dc0-caf4-421b-a739-a84fc7c52a21 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:07.073897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66702dc0-caf4-421b-a739-a84fc7c52a21 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:07.079022 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ddc2f-a84a-4a07-a240-fc05bf214179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:07.079780 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ddc2f-a84a-4a07-a240-fc05bf214179 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.080085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ddc2f-a84a-4a07-a240-fc05bf214179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.080361 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ddc2f-a84a-4a07-a240-fc05bf214179 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.081088 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ddc2f-a84a-4a07-a240-fc05bf214179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:07.082728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ddc2f-a84a-4a07-a240-fc05bf214179 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:07.093434 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1431/2875] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:07.097926 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689ffac-6ac7-4920-b531-374333332cc5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:07.108259 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689ffac-6ac7-4920-b531-374333332cc5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:07.114528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc24a09-3e26-469e-a36d-8138ccbebd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:07.115460 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc24a09-3e26-469e-a36d-8138ccbebd85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.115741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc24a09-3e26-469e-a36d-8138ccbebd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.116017 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc24a09-3e26-469e-a36d-8138ccbebd85 admin admin] PATH_INFO: `/v3/projects/b0cf3c93cea745b8b6207355f7f6508a/users/542d9f204b474c72b9abd91fa77e3357/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.133643 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc24a09-3e26-469e-a36d-8138ccbebd85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0cf3c93cea745b8b6207355f7f6508a, user_id=542d9f204b474c72b9abd91fa77e3357, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:07.136661 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc24a09-3e26-469e-a36d-8138ccbebd85 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:07.150698 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1445/2876] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:07 2022] PUT /identity/v3/projects/b0cf3c93cea745b8b6207355f7f6508a/users/542d9f204b474c72b9abd91fa77e3357/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:07.162551 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689c582-26e8-4f27-862f-4d406f63857d None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.162867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689c582-26e8-4f27-862f-4d406f63857d None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.163141 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689c582-26e8-4f27-862f-4d406f63857d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.194223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-c689c582-26e8-4f27-862f-4d406f63857d None None] MFA Rules not processed for user `542d9f204b474c72b9abd91fa77e3357`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:07.195501 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c689c582-26e8-4f27-862f-4d406f63857d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:07.267992 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1432/2877] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:07.272720 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc24a09-3e26-469e-a36d-8138ccbebd85 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:07.300295 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:07.353334 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc24a09-3e26-469e-a36d-8138ccbebd85 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:07.358440 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28220ea-9f41-4bcc-beba-02efeae650c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:07.359144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28220ea-9f41-4bcc-beba-02efeae650c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.359428 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28220ea-9f41-4bcc-beba-02efeae650c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.359707 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28220ea-9f41-4bcc-beba-02efeae650c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.360464 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28220ea-9f41-4bcc-beba-02efeae650c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:07.362387 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28220ea-9f41-4bcc-beba-02efeae650c7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:07.409465 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1446/2878] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:42:07 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:07.414698 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689c582-26e8-4f27-862f-4d406f63857d None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:07.427755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689c582-26e8-4f27-862f-4d406f63857d None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:07.487689 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a124b4-ceeb-4697-8065-39c313127a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:07.488379 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a124b4-ceeb-4697-8065-39c313127a6a admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.488663 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a124b4-ceeb-4697-8065-39c313127a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.488934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a124b4-ceeb-4697-8065-39c313127a6a admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.489625 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a124b4-ceeb-4697-8065-39c313127a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:07.491627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a124b4-ceeb-4697-8065-39c313127a6a admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:07.517248 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1433/2879] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:42:07 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:07.523924 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28220ea-9f41-4bcc-beba-02efeae650c7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:07.535003 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28220ea-9f41-4bcc-beba-02efeae650c7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:07.540996 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb70ed7-dd7e-45f0-871d-8042c7a7b4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:07.541686 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb70ed7-dd7e-45f0-871d-8042c7a7b4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.541963 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb70ed7-dd7e-45f0-871d-8042c7a7b4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.542220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb70ed7-dd7e-45f0-871d-8042c7a7b4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.542999 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb70ed7-dd7e-45f0-871d-8042c7a7b4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:07.544635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb70ed7-dd7e-45f0-871d-8042c7a7b4e5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:07.557140 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1447/2880] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:07.561712 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a124b4-ceeb-4697-8065-39c313127a6a admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:07.573231 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a124b4-ceeb-4697-8065-39c313127a6a admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:07.579418 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423fc8e-9f7b-419b-961a-03350ddc7be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:07.580726 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423fc8e-9f7b-419b-961a-03350ddc7be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.581007 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423fc8e-9f7b-419b-961a-03350ddc7be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.581272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423fc8e-9f7b-419b-961a-03350ddc7be7 admin admin] PATH_INFO: `/v3/projects/95becc3654a342fabedfead618b1c773/users/b23bbd66f973474d9c3175791eff3d1a/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.598270 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423fc8e-9f7b-419b-961a-03350ddc7be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95becc3654a342fabedfead618b1c773, user_id=b23bbd66f973474d9c3175791eff3d1a, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:07.601205 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423fc8e-9f7b-419b-961a-03350ddc7be7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:07.621442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1434/2881] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:07 2022] PUT /identity/v3/projects/95becc3654a342fabedfead618b1c773/users/b23bbd66f973474d9c3175791eff3d1a/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:07.627728 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb70ed7-dd7e-45f0-871d-8042c7a7b4e5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:07.638350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb70ed7-dd7e-45f0-871d-8042c7a7b4e5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:07.700835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa0754-2698-428a-af0b-cd7f18e33f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:07.701519 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa0754-2698-428a-af0b-cd7f18e33f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.701795 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa0754-2698-428a-af0b-cd7f18e33f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.702065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa0754-2698-428a-af0b-cd7f18e33f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.702832 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa0754-2698-428a-af0b-cd7f18e33f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:07.704735 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa0754-2698-428a-af0b-cd7f18e33f77 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:07.717030 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1448/2882] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:07.722319 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423fc8e-9f7b-419b-961a-03350ddc7be7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:07.733831 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423fc8e-9f7b-419b-961a-03350ddc7be7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:07.739215 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661e163-e8ff-4fd9-8323-dbb6a84b51f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:07.740107 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661e163-e8ff-4fd9-8323-dbb6a84b51f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.740382 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661e163-e8ff-4fd9-8323-dbb6a84b51f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.740651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661e163-e8ff-4fd9-8323-dbb6a84b51f8 admin admin] PATH_INFO: `/v3/projects/95becc3654a342fabedfead618b1c773/users/b23bbd66f973474d9c3175791eff3d1a/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.742834 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661e163-e8ff-4fd9-8323-dbb6a84b51f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95becc3654a342fabedfead618b1c773, user_id=b23bbd66f973474d9c3175791eff3d1a, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:07.745451 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661e163-e8ff-4fd9-8323-dbb6a84b51f8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:07.752594 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:07.760780 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1435/2883] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:07 2022] PUT /identity/v3/projects/95becc3654a342fabedfead618b1c773/users/b23bbd66f973474d9c3175791eff3d1a/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:07.773157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa872e94-9b0a-4422-b6f2-800cade1d4cc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.773574 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa872e94-9b0a-4422-b6f2-800cade1d4cc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.773976 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa872e94-9b0a-4422-b6f2-800cade1d4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.810044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-aa872e94-9b0a-4422-b6f2-800cade1d4cc None None] MFA Rules not processed for user `b23bbd66f973474d9c3175791eff3d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:07.811430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-aa872e94-9b0a-4422-b6f2-800cade1d4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:07.892713 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1449/2884] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:07.904545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661e163-e8ff-4fd9-8323-dbb6a84b51f8 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:07.906370 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-b661e163-e8ff-4fd9-8323-dbb6a84b51f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:07.974585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661e163-e8ff-4fd9-8323-dbb6a84b51f8 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:07.978512 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b96ca-8043-4dc0-ace2-3373597bfbaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:07.979185 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b96ca-8043-4dc0-ace2-3373597bfbaf service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:07.979458 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b96ca-8043-4dc0-ace2-3373597bfbaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:07.979728 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b96ca-8043-4dc0-ace2-3373597bfbaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:07.990158 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b96ca-8043-4dc0-ace2-3373597bfbaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:07.991629 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b96ca-8043-4dc0-ace2-3373597bfbaf service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:08.002175 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1436/2885] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:08.593838 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa872e94-9b0a-4422-b6f2-800cade1d4cc None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:08.613154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:08.666614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa872e94-9b0a-4422-b6f2-800cade1d4cc None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:08.679732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:08.680621 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:08.680896 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:08.681165 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] PATH_INFO: `/v3/users/542d9f204b474c72b9abd91fa77e3357` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:08.682389 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=542d9f204b474c72b9abd91fa77e3357)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:08.684384 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:08.737082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '542d9f204b474c72b9abd91fa77e3357'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:08.748875 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '542d9f204b474c72b9abd91fa77e3357'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:08.749668 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '542d9f204b474c72b9abd91fa77e3357'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:08.757943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '542d9f204b474c72b9abd91fa77e3357'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:08.758852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '542d9f204b474c72b9abd91fa77e3357'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:08.773848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1450/2886] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:42:08 2022] DELETE /identity/v3/users/542d9f204b474c72b9abd91fa77e3357 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:08.778648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8d43b-6bc3-4fc2-b98b-aaf3b5444346 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:08.778970 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8d43b-6bc3-4fc2-b98b-aaf3b5444346 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:08.779242 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8d43b-6bc3-4fc2-b98b-aaf3b5444346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:08.800681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-43b8d43b-6bc3-4fc2-b98b-aaf3b5444346 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:08.802341 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-43b8d43b-6bc3-4fc2-b98b-aaf3b5444346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:08.868080 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1437/2887] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:08.879526 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:08.881119 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:08.951468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eef170-78e3-4e48-8d99-1dd42a8d1c55 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:08.956799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b51cd9-935d-41e6-89ff-ee0b6d77c9a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:08.957493 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b51cd9-935d-41e6-89ff-ee0b6d77c9a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:08.957772 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b51cd9-935d-41e6-89ff-ee0b6d77c9a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:08.958044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b51cd9-935d-41e6-89ff-ee0b6d77c9a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:08.968396 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b51cd9-935d-41e6-89ff-ee0b6d77c9a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:08.970064 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b51cd9-935d-41e6-89ff-ee0b6d77c9a5 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:08.980239 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1451/2888] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:09.289865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8d43b-6bc3-4fc2-b98b-aaf3b5444346 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:09.291439 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-43b8d43b-6bc3-4fc2-b98b-aaf3b5444346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:09.306336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8d43b-6bc3-4fc2-b98b-aaf3b5444346 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:09.311160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:09.312135 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:09.312475 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:09.312807 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] PATH_INFO: `/v3/projects/b0cf3c93cea745b8b6207355f7f6508a` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:09.314166 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0cf3c93cea745b8b6207355f7f6508a)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:09.316210 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:09.368273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0cf3c93cea745b8b6207355f7f6508a', 'request_id': 'req-12b775dd-1788-4fc3-82f6-e329370ea0d7'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:09.377427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0cf3c93cea745b8b6207355f7f6508a', 'request_id': 'req-12b775dd-1788-4fc3-82f6-e329370ea0d7'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:09.385577 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0cf3c93cea745b8b6207355f7f6508a', 'request_id': 'req-12b775dd-1788-4fc3-82f6-e329370ea0d7'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:09.397576 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] The token cache is being invalidate because project b0cf3c93cea745b8b6207355f7f6508a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:42:09.397820 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:09.398500 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:09.400161 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1438/2889] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:42:09 2022] DELETE /identity/v3/projects/b0cf3c93cea745b8b6207355f7f6508a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:09.404287 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b51cd9-935d-41e6-89ff-ee0b6d77c9a5 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:09.415528 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b51cd9-935d-41e6-89ff-ee0b6d77c9a5 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:09.466349 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:09.496480 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08e3a8-3ac8-4b48-8e76-5dcc45af0bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:09.497314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08e3a8-3ac8-4b48-8e76-5dcc45af0bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:09.497598 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08e3a8-3ac8-4b48-8e76-5dcc45af0bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:09.497878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08e3a8-3ac8-4b48-8e76-5dcc45af0bd3 admin admin] PATH_INFO: `/v3/users/b23bbd66f973474d9c3175791eff3d1a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:09.499166 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08e3a8-3ac8-4b48-8e76-5dcc45af0bd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23bbd66f973474d9c3175791eff3d1a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:09.501439 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08e3a8-3ac8-4b48-8e76-5dcc45af0bd3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:09.560178 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-fa08e3a8-3ac8-4b48-8e76-5dcc45af0bd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23bbd66f973474d9c3175791eff3d1a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:09.573311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-fa08e3a8-3ac8-4b48-8e76-5dcc45af0bd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23bbd66f973474d9c3175791eff3d1a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:09.574061 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-fa08e3a8-3ac8-4b48-8e76-5dcc45af0bd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23bbd66f973474d9c3175791eff3d1a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:09.581840 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-fa08e3a8-3ac8-4b48-8e76-5dcc45af0bd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23bbd66f973474d9c3175791eff3d1a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:09.582546 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-fa08e3a8-3ac8-4b48-8e76-5dcc45af0bd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23bbd66f973474d9c3175791eff3d1a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:09.600910 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1452/2890] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:42:09 2022] DELETE /identity/v3/users/b23bbd66f973474d9c3175791eff3d1a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:09.910677 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:09.912151 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:09.926819 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b775dd-1788-4fc3-82f6-e329370ea0d7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:09.949407 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:09.998024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:09.998968 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:09.999245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:09.999489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] PATH_INFO: `/v3/projects/95becc3654a342fabedfead618b1c773` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.000812 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=95becc3654a342fabedfead618b1c773)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:10.002902 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:10.055293 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95becc3654a342fabedfead618b1c773', 'request_id': 'req-e79b0447-fefa-42b6-ba56-48f7bc665d6d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:10.065839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95becc3654a342fabedfead618b1c773', 'request_id': 'req-e79b0447-fefa-42b6-ba56-48f7bc665d6d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:10.073249 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95becc3654a342fabedfead618b1c773', 'request_id': 'req-e79b0447-fefa-42b6-ba56-48f7bc665d6d'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:10.084969 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] The token cache is being invalidate because project 95becc3654a342fabedfead618b1c773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80176) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:42:10.085256 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:10.085914 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-e79b0447-fefa-42b6-ba56-48f7bc665d6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:10.087716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1439/2891] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:42:09 2022] DELETE /identity/v3/projects/95becc3654a342fabedfead618b1c773 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:10.092744 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286be19-cbf8-4af4-b642-bef4a935c2af None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.093036 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286be19-cbf8-4af4-b642-bef4a935c2af None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.093315 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286be19-cbf8-4af4-b642-bef4a935c2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.114368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c286be19-cbf8-4af4-b642-bef4a935c2af None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:10.115839 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c286be19-cbf8-4af4-b642-bef4a935c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:10.185024 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1453/2892] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:10.194524 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465a5c5-cde1-44d7-85a8-0be9dc56baf3 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.194848 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465a5c5-cde1-44d7-85a8-0be9dc56baf3 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.195169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465a5c5-cde1-44d7-85a8-0be9dc56baf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.215340 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-c465a5c5-cde1-44d7-85a8-0be9dc56baf3 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:10.216579 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c465a5c5-cde1-44d7-85a8-0be9dc56baf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:10.225171 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1440/2893] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:10.229625 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286be19-cbf8-4af4-b642-bef4a935c2af None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:10.241262 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286be19-cbf8-4af4-b642-bef4a935c2af None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:10.246950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e290f8-d1a5-42f9-aee4-86972e9448fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:10.247552 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e290f8-d1a5-42f9-aee4-86972e9448fb admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.247812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e290f8-d1a5-42f9-aee4-86972e9448fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.248078 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e290f8-d1a5-42f9-aee4-86972e9448fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.248828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e290f8-d1a5-42f9-aee4-86972e9448fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:42:10.249132 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e290f8-d1a5-42f9-aee4-86972e9448fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:10.250801 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e290f8-d1a5-42f9-aee4-86972e9448fb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:10.266104 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1454/2894] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:42:10.271336 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465a5c5-cde1-44d7-85a8-0be9dc56baf3 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:10.283291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465a5c5-cde1-44d7-85a8-0be9dc56baf3 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:10.288395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445387c-6672-4579-9ac9-d2cb63f517a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:10.289079 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445387c-6672-4579-9ac9-d2cb63f517a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.289443 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445387c-6672-4579-9ac9-d2cb63f517a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.289660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445387c-6672-4579-9ac9-d2cb63f517a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.290378 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445387c-6672-4579-9ac9-d2cb63f517a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:10.292333 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445387c-6672-4579-9ac9-d2cb63f517a4 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:10.324821 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1441/2895] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:42:10 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:10.330105 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e290f8-d1a5-42f9-aee4-86972e9448fb admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:10.341065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e290f8-d1a5-42f9-aee4-86972e9448fb admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:10.399871 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:10.422189 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd8f93-31e2-4fbc-8829-e2abd524df4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:10.422998 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd8f93-31e2-4fbc-8829-e2abd524df4e admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.423274 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd8f93-31e2-4fbc-8829-e2abd524df4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.423543 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd8f93-31e2-4fbc-8829-e2abd524df4e admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.424237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd8f93-31e2-4fbc-8829-e2abd524df4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:10.426200 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd8f93-31e2-4fbc-8829-e2abd524df4e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:10.453020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1455/2896] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:42:10 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:10.458603 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445387c-6672-4579-9ac9-d2cb63f517a4 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:10.475323 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445387c-6672-4579-9ac9-d2cb63f517a4 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:10.481587 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc4e1-e049-4cee-ae68-d3146bcd6e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:10.482273 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc4e1-e049-4cee-ae68-d3146bcd6e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.482557 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc4e1-e049-4cee-ae68-d3146bcd6e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.482865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc4e1-e049-4cee-ae68-d3146bcd6e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.483607 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc4e1-e049-4cee-ae68-d3146bcd6e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:10.485223 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc4e1-e049-4cee-ae68-d3146bcd6e8a admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:10.497191 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1442/2897] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:10.501969 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd8f93-31e2-4fbc-8829-e2abd524df4e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:10.514588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd8f93-31e2-4fbc-8829-e2abd524df4e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:10.520835 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e0c4b-bc41-422e-8d6e-4c0938c7cb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:10.521575 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e0c4b-bc41-422e-8d6e-4c0938c7cb9a admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.521932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e0c4b-bc41-422e-8d6e-4c0938c7cb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.522221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e0c4b-bc41-422e-8d6e-4c0938c7cb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.523248 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e0c4b-bc41-422e-8d6e-4c0938c7cb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:10.525635 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e0c4b-bc41-422e-8d6e-4c0938c7cb9a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:10.537316 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1456/2898] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:10.541760 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc4e1-e049-4cee-ae68-d3146bcd6e8a admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:10.552822 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc4e1-e049-4cee-ae68-d3146bcd6e8a admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:10.557929 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3900d0-1ddc-4f6e-b718-ff2bb91fcd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:10.558855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3900d0-1ddc-4f6e-b718-ff2bb91fcd84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.559138 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3900d0-1ddc-4f6e-b718-ff2bb91fcd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.559445 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3900d0-1ddc-4f6e-b718-ff2bb91fcd84 admin admin] PATH_INFO: `/v3/projects/5188557e9f1d4af5a6b2594f6a654f20/users/41501016951444c8a746522c553ca6cd/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.578921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3900d0-1ddc-4f6e-b718-ff2bb91fcd84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5188557e9f1d4af5a6b2594f6a654f20, user_id=41501016951444c8a746522c553ca6cd, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:10.581700 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3900d0-1ddc-4f6e-b718-ff2bb91fcd84 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:10.598597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1443/2899] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:10 2022] PUT /identity/v3/projects/5188557e9f1d4af5a6b2594f6a654f20/users/41501016951444c8a746522c553ca6cd/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:10.610545 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c039c5-a1c8-40c7-aaef-be6809fd6695 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.611184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c039c5-a1c8-40c7-aaef-be6809fd6695 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.611184 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c039c5-a1c8-40c7-aaef-be6809fd6695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.640823 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-07c039c5-a1c8-40c7-aaef-be6809fd6695 None None] MFA Rules not processed for user `41501016951444c8a746522c553ca6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:10.642068 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-07c039c5-a1c8-40c7-aaef-be6809fd6695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:10.724336 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1457/2900] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:10.729662 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3900d0-1ddc-4f6e-b718-ff2bb91fcd84 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:10.793921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:10.809699 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3900d0-1ddc-4f6e-b718-ff2bb91fcd84 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:10.816272 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e69f7-cf61-46cb-9a82-161315566442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:10.816964 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e69f7-cf61-46cb-9a82-161315566442 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.817239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e69f7-cf61-46cb-9a82-161315566442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.817507 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e69f7-cf61-46cb-9a82-161315566442 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.818216 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e69f7-cf61-46cb-9a82-161315566442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:10.820169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e69f7-cf61-46cb-9a82-161315566442 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:10.850616 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1444/2901] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:42:10 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:42:10.854753 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c039c5-a1c8-40c7-aaef-be6809fd6695 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:10.867070 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c039c5-a1c8-40c7-aaef-be6809fd6695 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:10.933724 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a70d1a-0624-4916-90ca-257a9d9d2dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:10.934402 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a70d1a-0624-4916-90ca-257a9d9d2dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.934721 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a70d1a-0624-4916-90ca-257a9d9d2dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.934990 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a70d1a-0624-4916-90ca-257a9d9d2dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.935695 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a70d1a-0624-4916-90ca-257a9d9d2dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:10.937638 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a70d1a-0624-4916-90ca-257a9d9d2dc6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:10.959920 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1458/2902] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:42:10 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:10.964227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e69f7-cf61-46cb-9a82-161315566442 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:10.976094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e69f7-cf61-46cb-9a82-161315566442 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:10.981084 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252143aa-be41-4a6b-a177-31c0f145050b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:10.981775 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252143aa-be41-4a6b-a177-31c0f145050b admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:10.982055 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252143aa-be41-4a6b-a177-31c0f145050b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:10.982328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252143aa-be41-4a6b-a177-31c0f145050b admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:10.983094 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252143aa-be41-4a6b-a177-31c0f145050b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:10.984711 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252143aa-be41-4a6b-a177-31c0f145050b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:10.996425 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1445/2903] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:11.000681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a70d1a-0624-4916-90ca-257a9d9d2dc6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:11.012763 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a70d1a-0624-4916-90ca-257a9d9d2dc6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:11.018607 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9740da4-04d3-44ac-96af-0efa7ef80690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:11.019261 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9740da4-04d3-44ac-96af-0efa7ef80690 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:11.019540 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9740da4-04d3-44ac-96af-0efa7ef80690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:11.019812 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9740da4-04d3-44ac-96af-0efa7ef80690 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:11.020554 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9740da4-04d3-44ac-96af-0efa7ef80690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:11.022294 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9740da4-04d3-44ac-96af-0efa7ef80690 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:11.033749 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1459/2904] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:11.038389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252143aa-be41-4a6b-a177-31c0f145050b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:11.049496 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252143aa-be41-4a6b-a177-31c0f145050b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:11.055855 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56174-abda-4b27-9807-72eff1dddbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:11.056792 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56174-abda-4b27-9807-72eff1dddbad admin admin] REQUEST_METHOD: `PUT` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:11.057070 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56174-abda-4b27-9807-72eff1dddbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:11.057339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56174-abda-4b27-9807-72eff1dddbad admin admin] PATH_INFO: `/v3/projects/f3bf993fa8b74ccf86f7c59cf66b6691/users/cec8a719d0bf4d08aafd918f684df592/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:11.074853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f56174-abda-4b27-9807-72eff1dddbad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3bf993fa8b74ccf86f7c59cf66b6691, user_id=cec8a719d0bf4d08aafd918f684df592, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:11.077581 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f56174-abda-4b27-9807-72eff1dddbad admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:11.093819 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1446/2905] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:11 2022] PUT /identity/v3/projects/f3bf993fa8b74ccf86f7c59cf66b6691/users/cec8a719d0bf4d08aafd918f684df592/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:42:11.105247 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615aeaa-78ee-4605-917a-0a01f6295e9c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:11.105455 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615aeaa-78ee-4605-917a-0a01f6295e9c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:11.105737 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615aeaa-78ee-4605-917a-0a01f6295e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:11.138472 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d615aeaa-78ee-4605-917a-0a01f6295e9c None None] MFA Rules not processed for user `cec8a719d0bf4d08aafd918f684df592`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:11.140220 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d615aeaa-78ee-4605-917a-0a01f6295e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:11.171614 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:11.236701 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1460/2906] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:11.241528 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56174-abda-4b27-9807-72eff1dddbad admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:11.308618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56174-abda-4b27-9807-72eff1dddbad admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:11.314604 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b0909-7965-45f8-882f-f344550fdab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:11.315289 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b0909-7965-45f8-882f-f344550fdab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:11.315591 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b0909-7965-45f8-882f-f344550fdab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:11.315846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b0909-7965-45f8-882f-f344550fdab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:11.316545 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b0909-7965-45f8-882f-f344550fdab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:11.318468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b0909-7965-45f8-882f-f344550fdab0 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:11.349763 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1447/2907] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:42:11 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:11.354127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615aeaa-78ee-4605-917a-0a01f6295e9c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:11.367012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615aeaa-78ee-4605-917a-0a01f6295e9c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:11.436128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a07b5b-9484-4edb-824b-783d9e316715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:11.436814 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a07b5b-9484-4edb-824b-783d9e316715 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:11.437096 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a07b5b-9484-4edb-824b-783d9e316715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:11.437365 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a07b5b-9484-4edb-824b-783d9e316715 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:11.438058 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a07b5b-9484-4edb-824b-783d9e316715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:11.440342 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a07b5b-9484-4edb-824b-783d9e316715 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:11.463169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1461/2908] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:42:11 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:11.467481 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b0909-7965-45f8-882f-f344550fdab0 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:11.478871 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b0909-7965-45f8-882f-f344550fdab0 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:11.484755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80faddf-fe0f-4c82-98cc-20dbbd5029d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:11.485558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80faddf-fe0f-4c82-98cc-20dbbd5029d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:11.485841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80faddf-fe0f-4c82-98cc-20dbbd5029d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:11.486173 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80faddf-fe0f-4c82-98cc-20dbbd5029d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:11.486972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80faddf-fe0f-4c82-98cc-20dbbd5029d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:11.488557 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80faddf-fe0f-4c82-98cc-20dbbd5029d7 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:11.500456 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1448/2909] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:11.505063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a07b5b-9484-4edb-824b-783d9e316715 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:11.515590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a07b5b-9484-4edb-824b-783d9e316715 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:11.521561 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def087c-206f-4bd5-9bf8-ebafe0b41fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:11.522470 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def087c-206f-4bd5-9bf8-ebafe0b41fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:11.522777 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def087c-206f-4bd5-9bf8-ebafe0b41fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:11.523046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def087c-206f-4bd5-9bf8-ebafe0b41fc9 admin admin] PATH_INFO: `/v3/projects/66aeecc8d4fb4566b2d808552c7f44ff/users/e12b6e5173a0405697820aa1a8364df2/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:11.541605 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def087c-206f-4bd5-9bf8-ebafe0b41fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66aeecc8d4fb4566b2d808552c7f44ff, user_id=e12b6e5173a0405697820aa1a8364df2, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:11.544332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def087c-206f-4bd5-9bf8-ebafe0b41fc9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:11.563171 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1462/2910] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:11 2022] PUT /identity/v3/projects/66aeecc8d4fb4566b2d808552c7f44ff/users/e12b6e5173a0405697820aa1a8364df2/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:42:11.567863 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80faddf-fe0f-4c82-98cc-20dbbd5029d7 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:11.579880 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80faddf-fe0f-4c82-98cc-20dbbd5029d7 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:11.642685 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c560b-fd46-4ba4-bc1b-9259495a5644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:11.643290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c560b-fd46-4ba4-bc1b-9259495a5644 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:11.643570 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c560b-fd46-4ba4-bc1b-9259495a5644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:11.643843 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c560b-fd46-4ba4-bc1b-9259495a5644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:11.644597 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c560b-fd46-4ba4-bc1b-9259495a5644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:11.646212 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c560b-fd46-4ba4-bc1b-9259495a5644 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:11.656932 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1449/2911] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:11.662221 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def087c-206f-4bd5-9bf8-ebafe0b41fc9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:11.673326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def087c-206f-4bd5-9bf8-ebafe0b41fc9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:11.679482 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f73f5-1617-43f3-b82f-96bff84ae0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:11.680385 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f73f5-1617-43f3-b82f-96bff84ae0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:11.680657 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f73f5-1617-43f3-b82f-96bff84ae0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:11.680932 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f73f5-1617-43f3-b82f-96bff84ae0f4 admin admin] PATH_INFO: `/v3/projects/66aeecc8d4fb4566b2d808552c7f44ff/users/e12b6e5173a0405697820aa1a8364df2/roles/ecc9183a09db4d6b8ec98e96594d33ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:11.683169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f73f5-1617-43f3-b82f-96bff84ae0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66aeecc8d4fb4566b2d808552c7f44ff, user_id=e12b6e5173a0405697820aa1a8364df2, role_id=ecc9183a09db4d6b8ec98e96594d33ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:11.685849 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f73f5-1617-43f3-b82f-96bff84ae0f4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:11.697203 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1463/2912] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:11 2022] PUT /identity/v3/projects/66aeecc8d4fb4566b2d808552c7f44ff/users/e12b6e5173a0405697820aa1a8364df2/roles/ecc9183a09db4d6b8ec98e96594d33ba => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:11.708799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901af02-0845-4525-87b7-2c93ff49aff9 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:11.709086 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901af02-0845-4525-87b7-2c93ff49aff9 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:11.709385 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901af02-0845-4525-87b7-2c93ff49aff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:11.729759 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:11.739376 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-9901af02-0845-4525-87b7-2c93ff49aff9 None None] MFA Rules not processed for user `e12b6e5173a0405697820aa1a8364df2`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:11.741010 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9901af02-0845-4525-87b7-2c93ff49aff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:11.815008 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1450/2913] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:11.826844 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f73f5-1617-43f3-b82f-96bff84ae0f4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:11.829012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bd9f73f5-1617-43f3-b82f-96bff84ae0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:11.894930 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f73f5-1617-43f3-b82f-96bff84ae0f4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:11.898961 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53dd5d-d306-46a2-8ef5-8e28bb527106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:11.899619 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53dd5d-d306-46a2-8ef5-8e28bb527106 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:11.899897 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53dd5d-d306-46a2-8ef5-8e28bb527106 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:11.900204 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53dd5d-d306-46a2-8ef5-8e28bb527106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:11.930005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:11.965593 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53dd5d-d306-46a2-8ef5-8e28bb527106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:11.967120 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53dd5d-d306-46a2-8ef5-8e28bb527106 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:11.977042 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1464/2914] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:13.520615 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901af02-0845-4525-87b7-2c93ff49aff9 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:13.522227 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9901af02-0845-4525-87b7-2c93ff49aff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:13.537585 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901af02-0845-4525-87b7-2c93ff49aff9 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:13.541917 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb94c9-4405-4890-bc80-48a1b6775c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:13.542611 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb94c9-4405-4890-bc80-48a1b6775c05 service cinder] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:13.542899 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb94c9-4405-4890-bc80-48a1b6775c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:13.543163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb94c9-4405-4890-bc80-48a1b6775c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:13.607237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb94c9-4405-4890-bc80-48a1b6775c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:13.608731 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb94c9-4405-4890-bc80-48a1b6775c05 service cinder] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:13.619356 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1451/2915] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:14.242169 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53dd5d-d306-46a2-8ef5-8e28bb527106 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:14.254395 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53dd5d-d306-46a2-8ef5-8e28bb527106 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:14.259144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a9f32-1642-46bf-82f1-25e3061c1b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:14.259788 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9f32-1642-46bf-82f1-25e3061c1b8a service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:14.260091 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9f32-1642-46bf-82f1-25e3061c1b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:14.260345 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9f32-1642-46bf-82f1-25e3061c1b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:14.271059 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a9f32-1642-46bf-82f1-25e3061c1b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:14.272548 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a9f32-1642-46bf-82f1-25e3061c1b8a service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:14.282878 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1465/2916] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:18.223093 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb94c9-4405-4890-bc80-48a1b6775c05 service cinder] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:18.302824 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb94c9-4405-4890-bc80-48a1b6775c05 service cinder] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:18.308508 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:18.309304 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:18.309618 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:18.309885 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] PATH_INFO: `/v3/users/41501016951444c8a746522c553ca6cd` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:18.311239 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=41501016951444c8a746522c553ca6cd)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:18.313200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:18.369291 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:18.381109 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41501016951444c8a746522c553ca6cd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:18.393901 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41501016951444c8a746522c553ca6cd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:18.394688 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41501016951444c8a746522c553ca6cd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:18.395311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41501016951444c8a746522c553ca6cd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:18.402755 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41501016951444c8a746522c553ca6cd'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:18.418627 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1452/2917] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:42:18 2022] DELETE /identity/v3/users/41501016951444c8a746522c553ca6cd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:18.423399 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6beb90-056a-4384-9103-13f72e08113f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:18.424227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6beb90-056a-4384-9103-13f72e08113f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:18.424227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6beb90-056a-4384-9103-13f72e08113f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:18.445071 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ef6beb90-056a-4384-9103-13f72e08113f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:18.446286 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ef6beb90-056a-4384-9103-13f72e08113f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:18.518073 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1466/2918] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:42:18.528071 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:18.529708 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:18.598891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3d161-f2eb-4cac-b8e9-4c09da89c0ef admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:18.603311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dabb01-a3fc-4ce8-8169-a8742a571412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:18.603987 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dabb01-a3fc-4ce8-8169-a8742a571412 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:18.604220 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dabb01-a3fc-4ce8-8169-a8742a571412 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:18.604491 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dabb01-a3fc-4ce8-8169-a8742a571412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:18.614548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dabb01-a3fc-4ce8-8169-a8742a571412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:18.616030 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dabb01-a3fc-4ce8-8169-a8742a571412 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:18.625944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1453/2919] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:18.928494 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6beb90-056a-4384-9103-13f72e08113f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:18.930696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ef6beb90-056a-4384-9103-13f72e08113f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:18.947741 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6beb90-056a-4384-9103-13f72e08113f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:18.954855 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:18.956269 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:18.956542 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:18.956821 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] PATH_INFO: `/v3/projects/5188557e9f1d4af5a6b2594f6a654f20` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:18.958186 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5188557e9f1d4af5a6b2594f6a654f20)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:18.960383 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:19.014235 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5188557e9f1d4af5a6b2594f6a654f20', 'request_id': 'req-a0773605-be0f-4190-9e7d-120a781b4d75'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.025323 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5188557e9f1d4af5a6b2594f6a654f20', 'request_id': 'req-a0773605-be0f-4190-9e7d-120a781b4d75'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.031642 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5188557e9f1d4af5a6b2594f6a654f20', 'request_id': 'req-a0773605-be0f-4190-9e7d-120a781b4d75'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.044637 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] The token cache is being invalidate because project 5188557e9f1d4af5a6b2594f6a654f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:42:19.044899 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.045664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.047425 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1467/2920] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:42:18 2022] DELETE /identity/v3/projects/5188557e9f1d4af5a6b2594f6a654f20 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:19.052198 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dabb01-a3fc-4ce8-8169-a8742a571412 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:19.062765 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dabb01-a3fc-4ce8-8169-a8742a571412 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:19.133362 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:19.134162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:19.134442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:19.134743 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] PATH_INFO: `/v3/users/cec8a719d0bf4d08aafd918f684df592` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:19.136009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec8a719d0bf4d08aafd918f684df592)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:19.137958 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:19.187628 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec8a719d0bf4d08aafd918f684df592'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.196419 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:19.202009 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec8a719d0bf4d08aafd918f684df592'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.202818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec8a719d0bf4d08aafd918f684df592'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.203867 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec8a719d0bf4d08aafd918f684df592'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.212247 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec8a719d0bf4d08aafd918f684df592'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.229906 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1454/2921] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:42:19 2022] DELETE /identity/v3/users/cec8a719d0bf4d08aafd918f684df592 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:19.536940 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:19.538458 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:19.553014 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0773605-be0f-4190-9e7d-120a781b4d75 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:19.592631 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:19.622190 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:19.623100 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:19.623380 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:19.623652 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] PATH_INFO: `/v3/projects/f3bf993fa8b74ccf86f7c59cf66b6691` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:19.624965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3bf993fa8b74ccf86f7c59cf66b6691)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:19.627048 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:19.683051 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3bf993fa8b74ccf86f7c59cf66b6691', 'request_id': 'req-66835dca-ca2d-4b1d-a15d-062a59a24c83'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.692034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3bf993fa8b74ccf86f7c59cf66b6691', 'request_id': 'req-66835dca-ca2d-4b1d-a15d-062a59a24c83'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.698531 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3bf993fa8b74ccf86f7c59cf66b6691', 'request_id': 'req-66835dca-ca2d-4b1d-a15d-062a59a24c83'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.711782 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] The token cache is being invalidate because project f3bf993fa8b74ccf86f7c59cf66b6691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:42:19.712069 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.712799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.714461 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1468/2922] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:42:19 2022] DELETE /identity/v3/projects/f3bf993fa8b74ccf86f7c59cf66b6691 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 10:42:19.719063 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:19.720623 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:19.737442 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96da9e-db5c-45cb-a422-dd0bc2111c74 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:19.802190 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffb47d-f109-4270-938e-abdf3d54eb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:19.803122 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffb47d-f109-4270-938e-abdf3d54eb5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:19.803404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffb47d-f109-4270-938e-abdf3d54eb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:19.803677 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffb47d-f109-4270-938e-abdf3d54eb5d admin admin] PATH_INFO: `/v3/users/e12b6e5173a0405697820aa1a8364df2` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:19.804894 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffb47d-f109-4270-938e-abdf3d54eb5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e12b6e5173a0405697820aa1a8364df2)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:19.806875 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffb47d-f109-4270-938e-abdf3d54eb5d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:19.861290 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ddffb47d-f109-4270-938e-abdf3d54eb5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e12b6e5173a0405697820aa1a8364df2'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.877245 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ddffb47d-f109-4270-938e-abdf3d54eb5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e12b6e5173a0405697820aa1a8364df2'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.877870 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ddffb47d-f109-4270-938e-abdf3d54eb5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e12b6e5173a0405697820aa1a8364df2'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.880424 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ddffb47d-f109-4270-938e-abdf3d54eb5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e12b6e5173a0405697820aa1a8364df2'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.887972 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-ddffb47d-f109-4270-938e-abdf3d54eb5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e12b6e5173a0405697820aa1a8364df2'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:19.903528 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1455/2923] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:42:19 2022] DELETE /identity/v3/users/e12b6e5173a0405697820aa1a8364df2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:20.200524 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:20.202020 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:20.215920 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835dca-ca2d-4b1d-a15d-062a59a24c83 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:20.281445 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:20.282759 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:20.283046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:20.283430 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] PATH_INFO: `/v3/projects/66aeecc8d4fb4566b2d808552c7f44ff` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:20.285272 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66aeecc8d4fb4566b2d808552c7f44ff)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:20.288870 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:20.350035 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66aeecc8d4fb4566b2d808552c7f44ff', 'request_id': 'req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:20.359853 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66aeecc8d4fb4566b2d808552c7f44ff', 'request_id': 'req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:20.368377 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66aeecc8d4fb4566b2d808552c7f44ff', 'request_id': 'req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:20.376113 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:20.382046 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] The token cache is being invalidate because project 66aeecc8d4fb4566b2d808552c7f44ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:42:20.382632 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:20.383939 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c68d41fb-4926-4e1c-b71a-b46a2dc9d143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:20.385760 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1469/2924] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:42:20 2022] DELETE /identity/v3/projects/66aeecc8d4fb4566b2d808552c7f44ff => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:20.390976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11533ad9-c79b-4223-a95e-532b3eda4a8c None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:20.391473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11533ad9-c79b-4223-a95e-532b3eda4a8c None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:20.391653 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11533ad9-c79b-4223-a95e-532b3eda4a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:20.416145 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-11533ad9-c79b-4223-a95e-532b3eda4a8c None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:20.417779 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-11533ad9-c79b-4223-a95e-532b3eda4a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:20.482651 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:20.499808 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1456/2925] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:20.506379 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aeac78-c5f8-4740-9415-f749eb32b7d1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:20.506702 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aeac78-c5f8-4740-9415-f749eb32b7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:20.506968 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aeac78-c5f8-4740-9415-f749eb32b7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:20.527732 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-92aeac78-c5f8-4740-9415-f749eb32b7d1 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:20.529077 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-92aeac78-c5f8-4740-9415-f749eb32b7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:20.541852 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1470/2926] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:20.546200 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11533ad9-c79b-4223-a95e-532b3eda4a8c None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:20.557346 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11533ad9-c79b-4223-a95e-532b3eda4a8c None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:20.562847 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8eb8a-c57a-4e7a-9adf-ae5c9eb48e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:20.563540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8eb8a-c57a-4e7a-9adf-ae5c9eb48e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:20.563818 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8eb8a-c57a-4e7a-9adf-ae5c9eb48e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:20.564092 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8eb8a-c57a-4e7a-9adf-ae5c9eb48e5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:20.565190 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8eb8a-c57a-4e7a-9adf-ae5c9eb48e5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:42:20.565328 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8eb8a-c57a-4e7a-9adf-ae5c9eb48e5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:20.567024 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8eb8a-c57a-4e7a-9adf-ae5c9eb48e5b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:20.580548 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1457/2927] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 10:42:20.587394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aeac78-c5f8-4740-9415-f749eb32b7d1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:20.597986 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aeac78-c5f8-4740-9415-f749eb32b7d1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:20.604670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008f026-bd27-48a9-b6a9-b381589784d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:20.604670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008f026-bd27-48a9-b6a9-b381589784d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:20.604670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008f026-bd27-48a9-b6a9-b381589784d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:20.604670 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008f026-bd27-48a9-b6a9-b381589784d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:20.605162 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008f026-bd27-48a9-b6a9-b381589784d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:20.607127 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008f026-bd27-48a9-b6a9-b381589784d0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:20.637313 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1471/2928] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:42:20 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:20.642900 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8eb8a-c57a-4e7a-9adf-ae5c9eb48e5b admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:20.655891 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8eb8a-c57a-4e7a-9adf-ae5c9eb48e5b admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:20.727339 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3672d-c5be-4328-a4ca-d10974bf4c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:20.728105 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3672d-c5be-4328-a4ca-d10974bf4c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:20.728410 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3672d-c5be-4328-a4ca-d10974bf4c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:20.728681 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3672d-c5be-4328-a4ca-d10974bf4c14 admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:20.729390 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3672d-c5be-4328-a4ca-d10974bf4c14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:20.731387 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3672d-c5be-4328-a4ca-d10974bf4c14 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:20.757404 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1458/2929] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:42:20 2022] POST /identity/v3/users => generated 285 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 10:42:20.762138 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008f026-bd27-48a9-b6a9-b381589784d0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:20.773982 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008f026-bd27-48a9-b6a9-b381589784d0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:20.779266 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83edeea-771b-431e-a500-696ae858e623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:20.779955 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83edeea-771b-431e-a500-696ae858e623 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:20.780238 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83edeea-771b-431e-a500-696ae858e623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:20.780514 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83edeea-771b-431e-a500-696ae858e623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:20.781253 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83edeea-771b-431e-a500-696ae858e623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:20.783237 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83edeea-771b-431e-a500-696ae858e623 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:20.797065 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1472/2930] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:20.801632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3672d-c5be-4328-a4ca-d10974bf4c14 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:20.812184 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3672d-c5be-4328-a4ca-d10974bf4c14 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:20.817928 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1836414-b42d-47fb-bb5b-cdf40f0c58f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:20.818636 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1836414-b42d-47fb-bb5b-cdf40f0c58f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:20.818934 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1836414-b42d-47fb-bb5b-cdf40f0c58f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:20.819213 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1836414-b42d-47fb-bb5b-cdf40f0c58f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:20.819944 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1836414-b42d-47fb-bb5b-cdf40f0c58f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:20.821558 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1836414-b42d-47fb-bb5b-cdf40f0c58f2 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:20.833318 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1459/2931] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:20.839081 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83edeea-771b-431e-a500-696ae858e623 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:20.852110 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83edeea-771b-431e-a500-696ae858e623 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:20.857712 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ce906-07da-42ae-8981-b009a5ef55c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:20.859915 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ce906-07da-42ae-8981-b009a5ef55c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:20.860201 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ce906-07da-42ae-8981-b009a5ef55c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:20.860483 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ce906-07da-42ae-8981-b009a5ef55c2 admin admin] PATH_INFO: `/v3/projects/de170a9a337a41798644e2732353d0ea/users/bb34d9fbbd7a4ff681b92225e2fc2617/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:20.880324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ce906-07da-42ae-8981-b009a5ef55c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de170a9a337a41798644e2732353d0ea, user_id=bb34d9fbbd7a4ff681b92225e2fc2617, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:20.883431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ce906-07da-42ae-8981-b009a5ef55c2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:20.907413 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1473/2932] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:20 2022] PUT /identity/v3/projects/de170a9a337a41798644e2732353d0ea/users/bb34d9fbbd7a4ff681b92225e2fc2617/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:20.919660 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bf7e2-d556-4927-ba54-c6d445d9c5ee None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:20.920081 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bf7e2-d556-4927-ba54-c6d445d9c5ee None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:20.920365 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bf7e2-d556-4927-ba54-c6d445d9c5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:20.956038 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-933bf7e2-d556-4927-ba54-c6d445d9c5ee None None] MFA Rules not processed for user `bb34d9fbbd7a4ff681b92225e2fc2617`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:20.957276 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-933bf7e2-d556-4927-ba54-c6d445d9c5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:21.055606 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1460/2933] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:21.075646 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ce906-07da-42ae-8981-b009a5ef55c2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:21.077848 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f85ce906-07da-42ae-8981-b009a5ef55c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:21.155756 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ce906-07da-42ae-8981-b009a5ef55c2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:21.160318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e0bcc-c4c6-4179-ae01-e8621e223c09 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e850be24df24bcf972e6e31002014cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:21.160957 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e0bcc-c4c6-4179-ae01-e8621e223c09 service glance] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:21.161233 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e0bcc-c4c6-4179-ae01-e8621e223c09 service glance] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:21.161504 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e0bcc-c4c6-4179-ae01-e8621e223c09 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:21.176696 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e0bcc-c4c6-4179-ae01-e8621e223c09 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:21.179022 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e0bcc-c4c6-4179-ae01-e8621e223c09 service glance] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:21.192144 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1474/2934] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:31.051952 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:31.186170 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:52.648169 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bf7e2-d556-4927-ba54-c6d445d9c5ee None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:52.727650 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bf7e2-d556-4927-ba54-c6d445d9c5ee None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:52.732746 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:52.733632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:52.733907 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:52.734178 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] PATH_INFO: `/v3/users/bb34d9fbbd7a4ff681b92225e2fc2617` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:52.735473 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb34d9fbbd7a4ff681b92225e2fc2617)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:52.737412 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:52.793229 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb34d9fbbd7a4ff681b92225e2fc2617'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:52.807095 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb34d9fbbd7a4ff681b92225e2fc2617'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:52.807862 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb34d9fbbd7a4ff681b92225e2fc2617'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:52.808540 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb34d9fbbd7a4ff681b92225e2fc2617'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:52.816859 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb34d9fbbd7a4ff681b92225e2fc2617'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:52.833454 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1461/2935] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:42:52 2022] DELETE /identity/v3/users/bb34d9fbbd7a4ff681b92225e2fc2617 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:52.838314 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb230f1-7b52-4a1e-b9bc-7d812d183726 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:52.839374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb230f1-7b52-4a1e-b9bc-7d812d183726 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:52.839374 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb230f1-7b52-4a1e-b9bc-7d812d183726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:52.860590 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-bbb230f1-7b52-4a1e-b9bc-7d812d183726 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:52.862243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bbb230f1-7b52-4a1e-b9bc-7d812d183726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:52.930009 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1475/2936] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 10:42:52.940189 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:52.941655 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:53.004427 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1938ed8-4eaa-4d25-9d36-fbb68150378d admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:53.008881 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6cc6d-a401-4d93-93d2-edc0a2e23cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:53.009556 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6cc6d-a401-4d93-93d2-edc0a2e23cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:53.009930 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6cc6d-a401-4d93-93d2-edc0a2e23cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:53.010467 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6cc6d-a401-4d93-93d2-edc0a2e23cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:53.021267 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6cc6d-a401-4d93-93d2-edc0a2e23cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:53.022751 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6cc6d-a401-4d93-93d2-edc0a2e23cc3 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:53.033179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1462/2937] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:53.332664 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb230f1-7b52-4a1e-b9bc-7d812d183726 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:53.334332 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bbb230f1-7b52-4a1e-b9bc-7d812d183726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:53.350950 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb230f1-7b52-4a1e-b9bc-7d812d183726 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:53.356246 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:53.357126 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:53.358154 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:53.358432 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] PATH_INFO: `/v3/projects/de170a9a337a41798644e2732353d0ea` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:53.359951 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de170a9a337a41798644e2732353d0ea)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:53.362005 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:53.414388 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de170a9a337a41798644e2732353d0ea', 'request_id': 'req-109ac8f2-5c2d-4b48-8af3-fed8517d2577'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:53.423799 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de170a9a337a41798644e2732353d0ea', 'request_id': 'req-109ac8f2-5c2d-4b48-8af3-fed8517d2577'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:53.435962 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de170a9a337a41798644e2732353d0ea', 'request_id': 'req-109ac8f2-5c2d-4b48-8af3-fed8517d2577'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:53.448431 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] The token cache is being invalidate because project de170a9a337a41798644e2732353d0ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:42:53.448717 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:53.449468 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-109ac8f2-5c2d-4b48-8af3-fed8517d2577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:53.451180 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1476/2938] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:42:53 2022] DELETE /identity/v3/projects/de170a9a337a41798644e2732353d0ea => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:53.456286 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83f630-cf96-468c-85d2-10ba1513f85f None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:53.456566 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83f630-cf96-468c-85d2-10ba1513f85f None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:53.456839 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83f630-cf96-468c-85d2-10ba1513f85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:53.476799 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-4b83f630-cf96-468c-85d2-10ba1513f85f None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:53.478027 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-4b83f630-cf96-468c-85d2-10ba1513f85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:53.498589 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:53.550163 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1463/2939] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:53.558744 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e30f5e-a182-4466-a985-07b1d5ae30a6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:53.559025 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e30f5e-a182-4466-a985-07b1d5ae30a6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:53.559315 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e30f5e-a182-4466-a985-07b1d5ae30a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:53.579280 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a8e30f5e-a182-4466-a985-07b1d5ae30a6 None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:53.580722 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a8e30f5e-a182-4466-a985-07b1d5ae30a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:53.591964 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1477/2940] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:53.596580 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83f630-cf96-468c-85d2-10ba1513f85f None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:53.607147 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83f630-cf96-468c-85d2-10ba1513f85f None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:53.612226 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3ae1f-e19c-40b3-97d9-4d9759a7ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:53.612888 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3ae1f-e19c-40b3-97d9-4d9759a7ec45 admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:53.613194 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3ae1f-e19c-40b3-97d9-4d9759a7ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:53.613468 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3ae1f-e19c-40b3-97d9-4d9759a7ec45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:53.614204 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3ae1f-e19c-40b3-97d9-4d9759a7ec45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80176) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 10:42:53.614509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3ae1f-e19c-40b3-97d9-4d9759a7ec45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:53.616156 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3ae1f-e19c-40b3-97d9-4d9759a7ec45 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:53.629424 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1464/2941] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 10:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 10:42:53.634157 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e30f5e-a182-4466-a985-07b1d5ae30a6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:53.644350 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e30f5e-a182-4466-a985-07b1d5ae30a6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:53.649828 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78407d18-9f1c-4c13-b98d-d8f290440f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:53.651012 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407d18-9f1c-4c13-b98d-d8f290440f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:53.651311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407d18-9f1c-4c13-b98d-d8f290440f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:53.651566 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407d18-9f1c-4c13-b98d-d8f290440f6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:53.652255 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78407d18-9f1c-4c13-b98d-d8f290440f6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:53.654183 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78407d18-9f1c-4c13-b98d-d8f290440f6e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:53.684429 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1478/2942] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 10:42:53 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:53.691085 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3ae1f-e19c-40b3-97d9-4d9759a7ec45 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:53.703931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3ae1f-e19c-40b3-97d9-4d9759a7ec45 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:53.765696 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfb082-0f5d-4040-9d78-b8b717bf326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:53.766472 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfb082-0f5d-4040-9d78-b8b717bf326f admin admin] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:53.766766 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfb082-0f5d-4040-9d78-b8b717bf326f admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:53.767034 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfb082-0f5d-4040-9d78-b8b717bf326f admin admin] PATH_INFO: `/v3/users` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:53.767720 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfb082-0f5d-4040-9d78-b8b717bf326f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:53.769632 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfb082-0f5d-4040-9d78-b8b717bf326f admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:53.792237 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1465/2943] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 10:42:53 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 10:42:53.796557 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78407d18-9f1c-4c13-b98d-d8f290440f6e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:53.808063 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78407d18-9f1c-4c13-b98d-d8f290440f6e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:53.813570 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdc9bb-0d52-43c1-bf28-79872166f8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:53.814243 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdc9bb-0d52-43c1-bf28-79872166f8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:53.814535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdc9bb-0d52-43c1-bf28-79872166f8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:53.814830 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdc9bb-0d52-43c1-bf28-79872166f8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:53.815560 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdc9bb-0d52-43c1-bf28-79872166f8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:53.817223 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdc9bb-0d52-43c1-bf28-79872166f8c7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:53.828765 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1479/2944] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:53.832978 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfb082-0f5d-4040-9d78-b8b717bf326f admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:53.844014 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfb082-0f5d-4040-9d78-b8b717bf326f admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:53.850076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb813976-5335-41f0-a9fb-65621450b55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:53.850777 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb813976-5335-41f0-a9fb-65621450b55b admin admin] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:53.851076 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb813976-5335-41f0-a9fb-65621450b55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:53.851373 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb813976-5335-41f0-a9fb-65621450b55b admin admin] PATH_INFO: `/v3/roles` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:53.852140 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb813976-5335-41f0-a9fb-65621450b55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:53.853935 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb813976-5335-41f0-a9fb-65621450b55b admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:53.865179 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1466/2945] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 10:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 10:42:53.870285 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdc9bb-0d52-43c1-bf28-79872166f8c7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:53.881187 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdc9bb-0d52-43c1-bf28-79872166f8c7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:53.886669 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f447a-426e-40ac-adc7-93d9b8e501eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:53.887573 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f447a-426e-40ac-adc7-93d9b8e501eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:53.887854 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f447a-426e-40ac-adc7-93d9b8e501eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:53.888112 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f447a-426e-40ac-adc7-93d9b8e501eb admin admin] PATH_INFO: `/v3/projects/0f8706c0f5e64f91ae3841bad8f6d415/users/b95cdd172599479fafdc2c42e416fe40/roles/f7f18826d4b44c1284ac31e25088afbd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:53.904394 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f447a-426e-40ac-adc7-93d9b8e501eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8706c0f5e64f91ae3841bad8f6d415, user_id=b95cdd172599479fafdc2c42e416fe40, role_id=f7f18826d4b44c1284ac31e25088afbd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:53.907167 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f447a-426e-40ac-adc7-93d9b8e501eb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:53.937347 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1480/2946] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 10:42:53 2022] PUT /identity/v3/projects/0f8706c0f5e64f91ae3841bad8f6d415/users/b95cdd172599479fafdc2c42e416fe40/roles/f7f18826d4b44c1284ac31e25088afbd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:53.951682 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92220-5d1c-4efd-a162-06a7f6c85e13 None None] REQUEST_METHOD: `POST` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:53.951945 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92220-5d1c-4efd-a162-06a7f6c85e13 None None] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:53.952277 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92220-5d1c-4efd-a162-06a7f6c85e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:53.980234 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.auth.core [None req-fca92220-5d1c-4efd-a162-06a7f6c85e13 None None] MFA Rules not processed for user `b95cdd172599479fafdc2c42e416fe40`. Rule list: `[]` (Enabled: `True`). {{(pid=80176) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:53.981457 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-fca92220-5d1c-4efd-a162-06a7f6c85e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:54.062931 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1467/2947] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:54.075034 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f447a-426e-40ac-adc7-93d9b8e501eb admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:54.076900 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-cc1f447a-426e-40ac-adc7-93d9b8e501eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:54.101535 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:54.142326 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f447a-426e-40ac-adc7-93d9b8e501eb admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:54.146434 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5232c11-2191-47c3-aa59-7c3db81af0b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34813be7e8fd4fcfb5907bee6a465d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:54.147128 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5232c11-2191-47c3-aa59-7c3db81af0b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:54.147409 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5232c11-2191-47c3-aa59-7c3db81af0b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:54.147681 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5232c11-2191-47c3-aa59-7c3db81af0b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:54.157746 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5232c11-2191-47c3-aa59-7c3db81af0b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:54.159318 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5232c11-2191-47c3-aa59-7c3db81af0b3 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:54.169450 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1481/2948] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:54.196716 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca92220-5d1c-4efd-a162-06a7f6c85e13 None None] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:54.217160 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:42:54.267203 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca92220-5d1c-4efd-a162-06a7f6c85e13 None None] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:54.272225 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:54.273110 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:54.273395 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:54.273732 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] PATH_INFO: `/v3/users/b95cdd172599479fafdc2c42e416fe40` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:54.274976 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b95cdd172599479fafdc2c42e416fe40)` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:54.277162 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:54.334781 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b95cdd172599479fafdc2c42e416fe40'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:54.349988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b95cdd172599479fafdc2c42e416fe40'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:54.350921 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b95cdd172599479fafdc2c42e416fe40'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:54.351555 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b95cdd172599479fafdc2c42e416fe40'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:54.359617 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.notifications [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b95cdd172599479fafdc2c42e416fe40'} {{(pid=80176) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:54.385648 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1468/2949] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 10:42:54 2022] DELETE /identity/v3/users/b95cdd172599479fafdc2c42e416fe40 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:42:54.390591 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbccab-c392-4f4e-89c7-4fa980227e7a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:54.390943 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbccab-c392-4f4e-89c7-4fa980227e7a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:54.391227 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbccab-c392-4f4e-89c7-4fa980227e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:54.410044 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7bfbccab-c392-4f4e-89c7-4fa980227e7a None None] MFA Rules not processed for user `a021026b1cd24ba98923ebdb4a56f71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 10:42:54.411324 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7bfbccab-c392-4f4e-89c7-4fa980227e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:54.482604 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1482/2950] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 10:42:54.495865 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:54.497499 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:54.566853 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d043874-1149-4b47-b9d2-6e42cc276ed8 admin admin] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:54.573823 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d39c4-d823-4ad1-b619-728351abe474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c97b321bf854177bd2f310cf2958959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:54.574489 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d39c4-d823-4ad1-b619-728351abe474 service neutron] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:54.574797 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d39c4-d823-4ad1-b619-728351abe474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:54.575065 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d39c4-d823-4ad1-b619-728351abe474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:54.586311 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d39c4-d823-4ad1-b619-728351abe474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:54.587724 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d39c4-d823-4ad1-b619-728351abe474 service neutron] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:54.598509 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1469/2951] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:42:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:42:54.930880 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbccab-c392-4f4e-89c7-4fa980227e7a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:42:54.932606 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7bfbccab-c392-4f4e-89c7-4fa980227e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:42:54.945410 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbccab-c392-4f4e-89c7-4fa980227e7a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:42:54.951129 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a021026b1cd24ba98923ebdb4a56f71f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd79cb8b2f46e418d955fc3be77ed8c7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:42:54.952029 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:42:54.952311 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:42:54.952588 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] PATH_INFO: `/v3/projects/0f8706c0f5e64f91ae3841bad8f6d415` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:42:54.953820 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8706c0f5e64f91ae3841bad8f6d415)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:42:54.955810 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:42:55.013050 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8706c0f5e64f91ae3841bad8f6d415', 'request_id': 'req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:55.022198 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8706c0f5e64f91ae3841bad8f6d415', 'request_id': 'req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:55.028097 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8706c0f5e64f91ae3841bad8f6d415', 'request_id': 'req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:55.040368 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] The token cache is being invalidate because project 0f8706c0f5e64f91ae3841bad8f6d415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 10:42:55.040734 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:55.041437 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 10:42:55.046092 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1483/2952] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 10:42:54 2022] DELETE /identity/v3/projects/0f8706c0f5e64f91ae3841bad8f6d415 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 10:43:03.059046 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d39c4-d823-4ad1-b619-728351abe474 service neutron] Authenticating user token {{(pid=80176) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:43:03.060697 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-a70d39c4-d823-4ad1-b619-728351abe474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:43:03.105846 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:43:03.146206 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d39c4-d823-4ad1-b619-728351abe474 service neutron] Validating token access rules against request {{(pid=80176) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 10:43:03.210897 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce023f-cb91-4850-b41a-b9876c7a2894 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9bff6c855ea46feaa5526d2c27c9026', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4c742f196174c239f6a9c4aba775ee4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80176) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 10:43:03.211552 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce023f-cb91-4850-b41a-b9876c7a2894 service placement] REQUEST_METHOD: `GET` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 10:43:03.211841 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce023f-cb91-4850-b41a-b9876c7a2894 service placement] SCRIPT_NAME: `/identity` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 10:43:03.212114 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce023f-cb91-4850-b41a-b9876c7a2894 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80176) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 10:43:03.213988 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.fernet_utils [None req-9bce023f-cb91-4850-b41a-b9876c7a2894 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80176) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 10:43:03.225752 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce023f-cb91-4850-b41a-b9876c7a2894 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 10:43:03.227389 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce023f-cb91-4850-b41a-b9876c7a2894 service placement] RBAC: Authorization granted {{(pid=80176) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 10:43:03.237504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: [pid: 80176|app: 0|req: 1470/2953] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 10:43:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 10:43:05.036965 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:43:13.233504 dv-cinder-768105-16-iscsi devstack@keystone.service[80176]: DEBUG dbcounter [-] [80176] Writing DB stats keystone:SELECT=44 {{(pid=80176) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 10:48:08.998082 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 10:48:09.000015 dv-cinder-768105-16-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-43ca4922-c90a-4cd7-8e46-6ca7ab5a2904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}}